-- Logs begin at Mon 2022-08-08 01:18:33 UTC, end at Mon 2022-08-08 03:07:25 UTC. -- Aug 08 02:13:09.056885 dv-cinder-852261-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 02:13:09.067304 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 02:13:09 2022] *** Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: nodename: dv-cinder-852261-2-iscsi Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: machine: x86_64 Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: clock source: unix Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: pcre jit disabled Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: detected number of CPU cores: 4 Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: current working directory: / Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: detected binary path: /usr/bin/uwsgi-core Aug 08 02:13:09.069308 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: your processes number limit is 47680 Aug 08 02:13:09.069942 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: your memory page size is 4096 bytes Aug 08 02:13:09.069942 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: detected max file descriptor number: 2048 Aug 08 02:13:09.069942 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: lock engine: pthread robust mutexes Aug 08 02:13:09.069942 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: thunder lock: enabled Aug 08 02:13:09.069942 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 02:13:09.069942 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 02:13:09.107596 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: Python main interpreter initialized at 0x55d2563689c0 Aug 08 02:13:09.107596 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: python threads support enabled Aug 08 02:13:09.107596 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: your server socket listen backlog is limited to 100 connections Aug 08 02:13:09.107596 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: your mercy for graceful operations on workers is 90 seconds Aug 08 02:13:09.107925 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 02:13:09.108001 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: *** Operational MODE: preforking *** Aug 08 02:13:09.108270 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: *** uWSGI is running in multiple interpreter mode *** Aug 08 02:13:09.108270 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: spawned uWSGI master process (pid: 80255) Aug 08 02:13:09.108469 dv-cinder-852261-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 02:13:09.114380 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: spawned uWSGI worker 1 (pid: 80267, cores: 1) Aug 08 02:13:09.114380 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: spawned uWSGI worker 2 (pid: 80268, cores: 1) Aug 08 02:13:09.114380 dv-cinder-852261-2-iscsi devstack@keystone.service[80255]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 02:13:14.598772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:13:14.598772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: from cryptography.utils import int_from_bytes Aug 08 02:13:14.629138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:13:14.629138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: from cryptography.utils import int_from_bytes Aug 08 02:13:14.629611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:13:14.629611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: from cryptography.utils import int_from_bytes Aug 08 02:13:14.629742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:13:14.629742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: from cryptography.utils import int_from_bytes Aug 08 02:13:14.657370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:13:14.657564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:13:14.657721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:13:14.657814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] command line args: [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:13:14.657915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:13:14.657999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:13:14.658116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] command line args: [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:13:14.658256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ================================================================================ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:13:14.658403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:13:14.658495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] admin_token = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.658583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ================================================================================ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:13:14.658834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] admin_token = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.658934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_dir = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config_dir = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_source = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] debug = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659494 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] config_source = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] debug = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.659919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.660990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_config_append = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_config_append = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661502 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_dir = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_dir = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.661941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.662930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.663964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.664903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.665307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.665391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.665520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] publish_errors = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.665634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] publish_errors = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.665751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.665863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.666892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] standard_threads = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] standard_threads = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.667920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_journal = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_journal = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_json = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_json = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_stderr = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_stderr = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] use_syslog = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_syslog = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.668974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:13:14.669091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.669204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.669396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.669512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.669630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.669749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.669861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.669978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.670944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.671886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.672943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.673971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.674863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.675965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.676886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.677920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.678928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.679925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.680984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.681909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.682975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.683919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.684976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.685902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.686929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.687882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.688918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.689031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.689144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.689265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.689904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.690892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.691935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.692821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.external = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.693949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.external = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.password = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.694977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] auth.token = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.password = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.token = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.695898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.696940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.697990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.698920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.699939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.700160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.700273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.700457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.700583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.700583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.700798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.700910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.701941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.702978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.703887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.704885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.705070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.705741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.705917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.705917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.706148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.706297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.706432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.706561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.706679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.706793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.706906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.707965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.708991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.709998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.710955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.711984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.712899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713475 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.713936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.714913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715731 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.715970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.716892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.717933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.718983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.719903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720348 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.720858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.721991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.722965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.723913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.724848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.725947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.726927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727485 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.727959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.728978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.729954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.730956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.731939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732177 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.732945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.733877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.734912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.735949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.736999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.737915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.738927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.739966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.740890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.741919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.742034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.742147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.742269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80267) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:13:14.742474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.742474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.742704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.742825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.743056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.743294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.743793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.744029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.744274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.744511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.744754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.744989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.745236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.745473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.745716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.745946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.746185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.746448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:13:14.746646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:13:14.771926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.772221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.772367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.773026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.773259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.773485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.781942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.783202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.810990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.811214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.811297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.811505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.863785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.863911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.914636 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.914832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.914940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.915066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.978556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.978769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.978907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.978999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:14.979998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.003450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.003678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.003903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.004989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.005210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.016161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.016261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.028884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.029945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.030026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.030154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.030234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.030383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80267) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:13:15.046899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.047109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:13:15.047358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.047441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.047568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:13:15.047651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:13:15.047897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.047977 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.048093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:13:15.048173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.048364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.048447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.048624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.049220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.049572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.049876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.060547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.060825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.061095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.061176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.061373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.061468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.061640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.061915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.068971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.069545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.072044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.072329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.072581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.072662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.072864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.072959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.073145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.073228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.073404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.073484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.073674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.073764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.073942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.074021 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.074206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.074285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.074508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.074635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.074772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.074900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.075020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.075285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.097622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.098190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.099135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.099422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.099686 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.099970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.103995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.104281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.104511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.104593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.104792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.105070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.111649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.112108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.112188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.112579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.116320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.116568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.116649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.116858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.116940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.117143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.117224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.117429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.117510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.117714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.117809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.117995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.118076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.118262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.118343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.118834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.134979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.135225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.135308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.135503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.139219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.139444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.142345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.142601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.142862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.142944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.147223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.147312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.150345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.150478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.153476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.153573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.153773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.153856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.154069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.154150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.154377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.154472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.154658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.154742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.154944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.155035 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.170722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.170836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.171018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.171114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.171288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.171373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.171552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.171634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.179003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.179130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.180556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.180644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.180835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.180919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.183639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.183725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.183925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.184008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.184224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.184321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.184515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.184597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.184793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.184875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.196703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.196983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.197067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.197257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.197340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.197531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.197624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.197806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.197890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.198078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.198161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.198383 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.198468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.198657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.214724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.215165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.217644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.218104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.220604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.221158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.222259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.222750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.223760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.224042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.224224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.224331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.224517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.224800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.230673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.231207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.232257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.232809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.235187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.235476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.235727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.235817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.236018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.236101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.236925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.237355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.244786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.245075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.245418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.245664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.245960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.246245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.252857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80267) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.253136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.253409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.253657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:13:15.253742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.253932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.254014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.254201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.254287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.254493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.254578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.254770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.254863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.255047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.255129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.255337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.255419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.255617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.255708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.255917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.256005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.256189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.256271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.256455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80267) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.256543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.256776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.257060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.257323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:13:15.289259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d2563689c0 pid: 80267 (default app) Aug 08 02:13:15.290461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d2563689c0 pid: 80268 (default app) Aug 08 02:13:15.292483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687fd77-d61a-4413-b801-cfb7b342ad3a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:15.292767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687fd77-d61a-4413-b801-cfb7b342ad3a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:15.293035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687fd77-d61a-4413-b801-cfb7b342ad3a None None] PATH_INFO: `/v3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:15.297277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Mon Aug 8 02:13:15 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:23.735887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fec64-2f48-46bd-aa69-ec80d2d98024 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:23.736396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fec64-2f48-46bd-aa69-ec80d2d98024 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:23.736396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fec64-2f48-46bd-aa69-ec80d2d98024 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:23.740817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:23.746189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:23.746631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:23.746780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:23.816504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: INFO dbcounter [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] Registered counter for database keystone Aug 08 02:13:23.836570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG oslo_db.sqlalchemy.engines [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80267) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:13:23.842801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writer thread running {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:13:24.026272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG passlib.handlers.bcrypt [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80267) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 02:13:24.026272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG passlib.handlers.bcrypt [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80267) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 02:13:24.076601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:24.078203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:24.207095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 02:13:24.361332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fec64-2f48-46bd-aa69-ec80d2d98024 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.433325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: INFO dbcounter [None req-d98fec64-2f48-46bd-aa69-ec80d2d98024 None None] Registered counter for database keystone Aug 08 02:13:24.455108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG oslo_db.sqlalchemy.engines [None req-d98fec64-2f48-46bd-aa69-ec80d2d98024 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80268) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:13:24.461873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writer thread running {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:13:24.585215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fec64-2f48-46bd-aa69-ec80d2d98024 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.589610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c7c9e-b01e-420f-a2bd-b2b6c0c142c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.590229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7c9e-b01e-420f-a2bd-b2b6c0c142c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.590550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7c9e-b01e-420f-a2bd-b2b6c0c142c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.590827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7c9e-b01e-420f-a2bd-b2b6c0c142c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.599678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c7c9e-b01e-420f-a2bd-b2b6c0c142c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:24.653080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING keystone.common.rbac_enforcer.enforcer [None req-793c7c9e-b01e-420f-a2bd-b2b6c0c142c2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 02:13:24.653361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c7c9e-b01e-420f-a2bd-b2b6c0c142c2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:24.664006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:13:24.670067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:24.684948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967801d-bb02-48d0-9084-eed77c24d2c4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:24.689478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a346c-5165-4017-bed9-4f83e95de1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:24.690608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a346c-5165-4017-bed9-4f83e95de1bc admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:24.690897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a346c-5165-4017-bed9-4f83e95de1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:24.691172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a346c-5165-4017-bed9-4f83e95de1bc admin admin] PATH_INFO: `/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:24.694642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a346c-5165-4017-bed9-4f83e95de1bc admin admin] RBAC: Authorizing `identity:get_project(project_id=3e036fca31f44e57a9a6f1c124d31c6b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:24.744080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4d6a346c-5165-4017-bed9-4f83e95de1bc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 02:13:24.744422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a346c-5165-4017-bed9-4f83e95de1bc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:24.749073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Aug 8 02:13:24 2022] GET /identity/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:26.713812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bddfd7-c699-4d4e-9804-1749ce487340 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.714391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bddfd7-c699-4d4e-9804-1749ce487340 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.714391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bddfd7-c699-4d4e-9804-1749ce487340 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.718403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:26.723774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fee5e-be19-40ca-a47b-3fb35cf3d1fb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.724196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fee5e-be19-40ca-a47b-3fb35cf3d1fb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.724329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fee5e-be19-40ca-a47b-3fb35cf3d1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.745887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8d0fee5e-be19-40ca-a47b-3fb35cf3d1fb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:26.747453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8d0fee5e-be19-40ca-a47b-3fb35cf3d1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:26.756238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 02:13:26.777690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bddfd7-c699-4d4e-9804-1749ce487340 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.789890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bddfd7-c699-4d4e-9804-1749ce487340 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.794048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6bb83-d90c-4133-934f-1bc245320314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.794654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6bb83-d90c-4133-934f-1bc245320314 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.794930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6bb83-d90c-4133-934f-1bc245320314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.795200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6bb83-d90c-4133-934f-1bc245320314 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.802732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6bb83-d90c-4133-934f-1bc245320314 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.804491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6bb83-d90c-4133-934f-1bc245320314 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.812991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:13:26.818646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fee5e-be19-40ca-a47b-3fb35cf3d1fb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:26.827728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fee5e-be19-40ca-a47b-3fb35cf3d1fb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:26.832190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c12dc-b50b-4c24-a764-ba3bac70991d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:26.833436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c12dc-b50b-4c24-a764-ba3bac70991d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:26.833799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c12dc-b50b-4c24-a764-ba3bac70991d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:26.833992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c12dc-b50b-4c24-a764-ba3bac70991d admin admin] PATH_INFO: `/v3/users/c5565152ee164c0da536fef7f9cd3d6c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:26.835338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c12dc-b50b-4c24-a764-ba3bac70991d admin admin] RBAC: Authorizing `identity:get_user(user_id=c5565152ee164c0da536fef7f9cd3d6c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:26.837180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c12dc-b50b-4c24-a764-ba3bac70991d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:26.838715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Mon Aug 8 02:13:26 2022] GET /identity/v3/users/c5565152ee164c0da536fef7f9cd3d6c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:28.873733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5271bca-c68a-4256-8425-d30c1cf81d86 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.874272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5271bca-c68a-4256-8425-d30c1cf81d86 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.874272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5271bca-c68a-4256-8425-d30c1cf81d86 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.875645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:28.880811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4aff-c8fd-4287-a8b4-b14dea3d6856 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.881215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4aff-c8fd-4287-a8b4-b14dea3d6856 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.881468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f4aff-c8fd-4287-a8b4-b14dea3d6856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.898997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-189f4aff-c8fd-4287-a8b4-b14dea3d6856 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:28.900155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-189f4aff-c8fd-4287-a8b4-b14dea3d6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:28.938377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e58010-f714-43a8-82db-31fcd92686f7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.938756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e58010-f714-43a8-82db-31fcd92686f7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.939097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e58010-f714-43a8-82db-31fcd92686f7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.940489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:28.945562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:28.945847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:28.946123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:28.984254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:28.989772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG passlib.handlers.bcrypt [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80268) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 02:13:28.990212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG passlib.handlers.bcrypt [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80268) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 02:13:29.007698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f4aff-c8fd-4287-a8b4-b14dea3d6856 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.030305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.035562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:29.037667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:29.111640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f4aff-c8fd-4287-a8b4-b14dea3d6856 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.137295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:29.146198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.146848 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.147128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.147402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.158206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:29.159361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:29.167446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:29.168358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:29.169297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:29.174259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.254899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.261442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0a902-479b-41f6-913b-ef5c733cc06c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.263844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.273114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f07a7-adf9-4f06-af56-e0a8b7adc640 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.299180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.300060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.300313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.300583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.307536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.308133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.308417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.308690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.309349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:29.309651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.311084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.322856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:29.328322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.368273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.369811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.472756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:29.479034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.482162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.538694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f596e0-1953-485e-8c41-51dd538ffb3c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.572639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.573260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.573540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.573807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] PATH_INFO: `/v3/users/c5565152ee164c0da536fef7f9cd3d6c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.574953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] RBAC: Authorizing `identity:get_user(user_id=c5565152ee164c0da536fef7f9cd3d6c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.576387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.577927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/users/c5565152ee164c0da536fef7f9cd3d6c => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:29.581657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727fb1e-74b7-4659-853c-531a8b918c2e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.583709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.597657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.616080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.616927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.617213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.617481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.635526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.636984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:29.654589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:29.655639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:29.656644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:29.656644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:29.656644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:29.656644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:29.663508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.687985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cffe7-ead1-45d6-b7c1-e2e2c5f2a7b5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.717933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.722082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.722746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.723081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.723880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.725028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.726520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.744985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:29.744985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:29.762412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c08eba-900a-4490-a6ff-48f688c39ec2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.796205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc193df2-adea-434d-a3ed-28b5af4b10db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.796808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc193df2-adea-434d-a3ed-28b5af4b10db None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.797092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc193df2-adea-434d-a3ed-28b5af4b10db None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.797370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc193df2-adea-434d-a3ed-28b5af4b10db None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.798015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc193df2-adea-434d-a3ed-28b5af4b10db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:29.798327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc193df2-adea-434d-a3ed-28b5af4b10db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.799742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc193df2-adea-434d-a3ed-28b5af4b10db None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.813569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:29.817617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 10/20] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:29.832876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6e33e-8957-474c-9df6-782a07dad937 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:29.872159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77823da1-7b27-4cd0-8c95-3b6fe96c8481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:29.872791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823da1-7b27-4cd0-8c95-3b6fe96c8481 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:29.873071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823da1-7b27-4cd0-8c95-3b6fe96c8481 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:29.873342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77823da1-7b27-4cd0-8c95-3b6fe96c8481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:29.874087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77823da1-7b27-4cd0-8c95-3b6fe96c8481 None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.domain.id=default, user.id=c5565152ee164c0da536fef7f9cd3d6c) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:29.874421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77823da1-7b27-4cd0-8c95-3b6fe96c8481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:29.875918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77823da1-7b27-4cd0-8c95-3b6fe96c8481 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:29.883770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 11/21] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Aug 8 02:13:29 2022] GET /identity/v3/role_assignments?user.id=c5565152ee164c0da536fef7f9cd3d6c&scope.domain.id=default&role.id=1b9fb565648c49388166fe130739891f => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:32.333908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad96b3b-b9d2-44fa-89e3-9cb70e86aff4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.334964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad96b3b-b9d2-44fa-89e3-9cb70e86aff4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.335469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad96b3b-b9d2-44fa-89e3-9cb70e86aff4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.337027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 11/22] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:32.342265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8026-5045-47e5-b026-840359d241c7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.343112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8026-5045-47e5-b026-840359d241c7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.343583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8026-5045-47e5-b026-840359d241c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.359925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ada413-b96d-4022-b383-925d709679cb None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.360545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ada413-b96d-4022-b383-925d709679cb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.361047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ada413-b96d-4022-b383-925d709679cb None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.362915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 12/23] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:32.367716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f71fb5-66bf-41d0-b3d3-80df86d38dbf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.368497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f71fb5-66bf-41d0-b3d3-80df86d38dbf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.368989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f71fb5-66bf-41d0-b3d3-80df86d38dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.378589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ed2a8026-5045-47e5-b026-840359d241c7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:32.381154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ed2a8026-5045-47e5-b026-840359d241c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:32.400444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-79f71fb5-66bf-41d0-b3d3-80df86d38dbf None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:32.401986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-79f71fb5-66bf-41d0-b3d3-80df86d38dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:32.523732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 12/24] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:32.540964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 13/25] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:32.563289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8026-5045-47e5-b026-840359d241c7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.591675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f71fb5-66bf-41d0-b3d3-80df86d38dbf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:32.666752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.725789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f71fb5-66bf-41d0-b3d3-80df86d38dbf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.735352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:32.765737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d08d31-fa67-4737-910c-c8dc4db890d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.766846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d08d31-fa67-4737-910c-c8dc4db890d8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.767327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d08d31-fa67-4737-910c-c8dc4db890d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.767813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d08d31-fa67-4737-910c-c8dc4db890d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.767932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8026-5045-47e5-b026-840359d241c7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:32.783759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d08d31-fa67-4737-910c-c8dc4db890d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-18d08d31-fa67-4737-910c-c8dc4db890d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:32.790945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:32.791842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:32.792571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:32.795368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 14/26] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:32.796981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04915-5949-4552-b786-cd355735c84a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.797474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04915-5949-4552-b786-cd355735c84a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.797943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04915-5949-4552-b786-cd355735c84a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.799421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 15/27] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:32.800640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710968-61f5-4316-b7be-23b4c8d3f241 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.801130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710968-61f5-4316-b7be-23b4c8d3f241 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.801647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d710968-61f5-4316-b7be-23b4c8d3f241 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.803436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 16/28] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:32.807847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37ded-8780-4ba9-8393-57723d490a24 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.808614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37ded-8780-4ba9-8393-57723d490a24 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.809096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37ded-8780-4ba9-8393-57723d490a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.823470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9a8ee-ae42-48bb-ad39-c93d50420055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:32.824095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9a8ee-ae42-48bb-ad39-c93d50420055 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:32.824380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9a8ee-ae42-48bb-ad39-c93d50420055 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:32.824657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9a8ee-ae42-48bb-ad39-c93d50420055 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:32.825632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9a8ee-ae42-48bb-ad39-c93d50420055 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:32.827685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9a8ee-ae42-48bb-ad39-c93d50420055 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:32.835485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f5e37ded-8780-4ba9-8393-57723d490a24 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:32.837040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f5e37ded-8780-4ba9-8393-57723d490a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:33.022046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 17/29] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:33.023282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37ded-8780-4ba9-8393-57723d490a24 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.040619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 13/30] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 02:13:32 2022] POST /identity/v3/roles => generated 219 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:33.042600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5139-13a3-4050-8a8b-01e8da2fa177 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.043163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5139-13a3-4050-8a8b-01e8da2fa177 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.043656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d5139-13a3-4050-8a8b-01e8da2fa177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.065197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d35d5139-13a3-4050-8a8b-01e8da2fa177 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:33.067796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d35d5139-13a3-4050-8a8b-01e8da2fa177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:33.082029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.199948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 14/31] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:33.200979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5139-13a3-4050-8a8b-01e8da2fa177 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.207795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37ded-8780-4ba9-8393-57723d490a24 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.277457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.278080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.278648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.278934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.280022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.280335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.282171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.297537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.302516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 18/32] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.303320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.357396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d5139-13a3-4050-8a8b-01e8da2fa177 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.394558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db768f7a-8e36-4e7f-a0b3-344558ff3e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.395729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768f7a-8e36-4e7f-a0b3-344558ff3e40 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.396011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768f7a-8e36-4e7f-a0b3-344558ff3e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.396361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768f7a-8e36-4e7f-a0b3-344558ff3e40 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.396944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db768f7a-8e36-4e7f-a0b3-344558ff3e40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.398653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db768f7a-8e36-4e7f-a0b3-344558ff3e40 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.421509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.425268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba1051-0389-4c8c-aecd-1ea28f4c1409 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.458423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 15/33] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 02:13:33 2022] POST /identity/v3/roles => generated 225 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:33.459913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7443e-677b-44c0-ad60-ebd7ec9e9fcf None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.460200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7443e-677b-44c0-ad60-ebd7ec9e9fcf None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.460483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7443e-677b-44c0-ad60-ebd7ec9e9fcf None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.461338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeef39d-cc1d-4a9f-9b1f-430d1d694db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.461985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef39d-cc1d-4a9f-9b1f-430d1d694db2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.462265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef39d-cc1d-4a9f-9b1f-430d1d694db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.462572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef39d-cc1d-4a9f-9b1f-430d1d694db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.468025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 16/34] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:33.469357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7443e-677b-44c0-ad60-ebd7ec9e9fcf None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.482049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeef39d-cc1d-4a9f-9b1f-430d1d694db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.485179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeef39d-cc1d-4a9f-9b1f-430d1d694db2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5aeef39d-cc1d-4a9f-9b1f-430d1d694db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:13:33.509943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:33.511071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:13:33.511867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:13:33.511867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:13:33.511867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:33.511867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 19/35] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:33.517535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07bc04-0ef5-4452-931b-ecb720eb503a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.517535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07bc04-0ef5-4452-931b-ecb720eb503a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.517535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07bc04-0ef5-4452-931b-ecb720eb503a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.541990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5b07bc04-0ef5-4452-931b-ecb720eb503a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:33.543565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5b07bc04-0ef5-4452-931b-ecb720eb503a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:33.617108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7443e-677b-44c0-ad60-ebd7ec9e9fcf None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.624618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 20/36] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:33.625643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07bc04-0ef5-4452-931b-ecb720eb503a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.647268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.659861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.660872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.661272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.661858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] PATH_INFO: `/v3/users/c5565152ee164c0da536fef7f9cd3d6c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.663390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] RBAC: Authorizing `identity:get_user(user_id=c5565152ee164c0da536fef7f9cd3d6c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.664966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.666997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 17/37] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/users/c5565152ee164c0da536fef7f9cd3d6c => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.667713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.721282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.750433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07bc04-0ef5-4452-931b-ecb720eb503a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.789894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.790543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.790821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.791093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.791736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:33.792045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.793427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.799756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7eb05-de40-4042-ac48-3d651e4904c0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.806203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 21/38] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.807075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.835627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af5f6e-2807-49f6-a6bc-78f67923dfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.836232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af5f6e-2807-49f6-a6bc-78f67923dfa5 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.836519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af5f6e-2807-49f6-a6bc-78f67923dfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.836790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af5f6e-2807-49f6-a6bc-78f67923dfa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.837452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af5f6e-2807-49f6-a6bc-78f67923dfa5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.838924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af5f6e-2807-49f6-a6bc-78f67923dfa5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.904192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 18/39] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 02:13:33 2022] POST /identity/v3/roles => generated 223 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:33.905425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bb67-9e3c-4e4d-893e-41c1ad7182c7 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.905695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bb67-9e3c-4e4d-893e-41c1ad7182c7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.905975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bb67-9e3c-4e4d-893e-41c1ad7182c7 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.907246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 19/40] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:33.907951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9bb67-9e3c-4e4d-893e-41c1ad7182c7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:33.920779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a1819-589e-4e39-ba4f-d5adddddce8b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:33.965054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb8fa1-bd2b-46c4-9fde-b208a0eaa224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:33.965701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb8fa1-bd2b-46c4-9fde-b208a0eaa224 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.965990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb8fa1-bd2b-46c4-9fde-b208a0eaa224 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.966265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb8fa1-bd2b-46c4-9fde-b208a0eaa224 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.967769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb8fa1-bd2b-46c4-9fde-b208a0eaa224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:33.969243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb8fa1-bd2b-46c4-9fde-b208a0eaa224 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:33.970912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 22/41] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:33.972161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d088-68b8-472c-a233-ef085b6b9ba1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:33.972373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d088-68b8-472c-a233-ef085b6b9ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:33.972646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d088-68b8-472c-a233-ef085b6b9ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:33.990559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:33.992405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-47a2d088-68b8-472c-a233-ef085b6b9ba1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:33.993930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-47a2d088-68b8-472c-a233-ef085b6b9ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:33.996212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.046022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9bb67-9e3c-4e4d-893e-41c1ad7182c7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.074715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 23/42] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:34.075743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2d088-68b8-472c-a233-ef085b6b9ba1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.097127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.098519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.098805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.099144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.099793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.101718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.181205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2d088-68b8-472c-a233-ef085b6b9ba1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.185224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 20/43] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:13:33 2022] POST /identity/v3/projects => generated 301 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:34.186052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.217708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa7c8-4fd7-4c9a-8466-fff34d409dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.218414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa7c8-4fd7-4c9a-8466-fff34d409dc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.218696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa7c8-4fd7-4c9a-8466-fff34d409dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.218967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa7c8-4fd7-4c9a-8466-fff34d409dc5 None admin] PATH_INFO: `/v3/domains/default/users/c5565152ee164c0da536fef7f9cd3d6c/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.221219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7aa7c8-4fd7-4c9a-8466-fff34d409dc5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c5565152ee164c0da536fef7f9cd3d6c, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.222801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7aa7c8-4fd7-4c9a-8466-fff34d409dc5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.289301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.317945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf1a2-c039-4feb-ac44-ddfddd59027a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.320902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:34.357482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.358125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.358403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.358678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.359798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.361412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.363028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 21/44] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:34.370520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:34.405828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 24/45] 10.222.0.15 () {58 vars in 1361 bytes} [Mon Aug 8 02:13:34 2022] PUT /identity/v3/domains/default/users/c5565152ee164c0da536fef7f9cd3d6c/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:34.477756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dc1db-77e8-4a89-9b05-59588632f25a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:34.512633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e824ab-fd25-4247-a6c3-7f0a5cb77e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:34.513247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e824ab-fd25-4247-a6c3-7f0a5cb77e16 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:34.513526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e824ab-fd25-4247-a6c3-7f0a5cb77e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:34.513796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e824ab-fd25-4247-a6c3-7f0a5cb77e16 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:34.514486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e824ab-fd25-4247-a6c3-7f0a5cb77e16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:34.515975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e824ab-fd25-4247-a6c3-7f0a5cb77e16 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:34.596809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 22/46] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:13:34 2022] POST /identity/v3/projects => generated 312 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:36.388386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8ea9c-bb64-4e9d-a4bf-0e7e3e39273a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.388899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8ea9c-bb64-4e9d-a4bf-0e7e3e39273a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.389060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8ea9c-bb64-4e9d-a4bf-0e7e3e39273a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.393375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 25/47] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:36.399077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1e627-133d-48ba-860f-3407e0a9fe08 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.399500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1e627-133d-48ba-860f-3407e0a9fe08 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.399637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1e627-133d-48ba-860f-3407e0a9fe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.418060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-72e1e627-133d-48ba-860f-3407e0a9fe08 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:36.419280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-72e1e627-133d-48ba-860f-3407e0a9fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:36.436704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.484666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 23/48] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:36.508091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8ea9c-bb64-4e9d-a4bf-0e7e3e39273a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.600277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99847dd-3295-4b8c-b6b8-2d77de9faa37 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.600568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99847dd-3295-4b8c-b6b8-2d77de9faa37 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.600849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99847dd-3295-4b8c-b6b8-2d77de9faa37 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.602200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 24/49] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:36.607239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7abeb-4324-426c-92af-cb77c8479bc8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.607530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7abeb-4324-426c-92af-cb77c8479bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.607805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7abeb-4324-426c-92af-cb77c8479bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.610052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8ea9c-bb64-4e9d-a4bf-0e7e3e39273a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.625922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d0c7abeb-4324-426c-92af-cb77c8479bc8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:36.627148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d0c7abeb-4324-426c-92af-cb77c8479bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:36.643727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.644290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.644597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.645399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.655464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:36.660218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:36.661889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:36.662840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 26/50] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:36.668570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.693989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 25/51] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:36.716677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7abeb-4324-426c-92af-cb77c8479bc8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.730117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.772429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f5743-7a23-4f3a-ac13-1386c1ff430f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.798841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.805862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.806492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.806782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.807060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.807735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:36.808043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.809641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.814555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7abeb-4324-426c-92af-cb77c8479bc8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.818423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 27/52] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:36.823865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.850758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.851379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.851660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.851926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.853032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.854531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.856070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:36.861536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:36.921763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105e44a-3941-4987-92aa-3dea205433c3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.941526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:36.955712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:36.956340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:36.956622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:36.956906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] PATH_INFO: `/v3/users/c5565152ee164c0da536fef7f9cd3d6c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:36.958802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] RBAC: Authorizing `identity:get_user(user_id=c5565152ee164c0da536fef7f9cd3d6c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:36.960371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:36.963687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f5fad-62f8-4ea5-9ed6-14663d7d953a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:36.966547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 28/54] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/users/c5565152ee164c0da536fef7f9cd3d6c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:36.967470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.001258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.001864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.002143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.002454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.003119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.005084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.065190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67060cdd-6e0d-434b-ac1e-b2af08d205f4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.081806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 27/55] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Aug 8 02:13:36 2022] POST /identity/v3/projects => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:13:37.101108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1effaf-9105-49cb-a9e2-b977bb7ee534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.101765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1effaf-9105-49cb-a9e2-b977bb7ee534 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.102050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1effaf-9105-49cb-a9e2-b977bb7ee534 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.102322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1effaf-9105-49cb-a9e2-b977bb7ee534 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.103885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1effaf-9105-49cb-a9e2-b977bb7ee534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.105395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1effaf-9105-49cb-a9e2-b977bb7ee534 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.107121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 29/56] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:37.113306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:37.117372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:37.220277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a86bb2-5119-4935-9017-6243c976fd15 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:37.256185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777fe65-edcd-42bc-af6e-0ce49bc0e8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:37.256792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777fe65-edcd-42bc-af6e-0ce49bc0e8bb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:37.257071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777fe65-edcd-42bc-af6e-0ce49bc0e8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:37.257348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777fe65-edcd-42bc-af6e-0ce49bc0e8bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:37.258068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777fe65-edcd-42bc-af6e-0ce49bc0e8bb None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.domain.id=default, user.id=c5565152ee164c0da536fef7f9cd3d6c) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:37.258404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777fe65-edcd-42bc-af6e-0ce49bc0e8bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:37.259866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777fe65-edcd-42bc-af6e-0ce49bc0e8bb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:37.267048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 28/57] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Aug 8 02:13:37 2022] GET /identity/v3/role_assignments?user.id=c5565152ee164c0da536fef7f9cd3d6c&scope.domain.id=default&role.id=1b9fb565648c49388166fe130739891f => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.097291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3b82-3ebd-4b34-a561-9c2a7e1c3e41 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.098382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3b82-3ebd-4b34-a561-9c2a7e1c3e41 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.098382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3b82-3ebd-4b34-a561-9c2a7e1c3e41 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.099755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 30/58] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:39.104892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67019e-fc93-42a1-967c-6f9317660283 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.105281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67019e-fc93-42a1-967c-6f9317660283 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.105451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67019e-fc93-42a1-967c-6f9317660283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.123340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ce67019e-fc93-42a1-967c-6f9317660283 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:39.124536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ce67019e-fc93-42a1-967c-6f9317660283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:39.185223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.190303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 29/59] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:39.215033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c3b82-3ebd-4b34-a561-9c2a7e1c3e41 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.272169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:39.315885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c3b82-3ebd-4b34-a561-9c2a7e1c3e41 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.348805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e728e1-c155-414d-9384-52c9b1554f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.349412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e728e1-c155-414d-9384-52c9b1554f71 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.349694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e728e1-c155-414d-9384-52c9b1554f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.349968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e728e1-c155-414d-9384-52c9b1554f71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.351101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e728e1-c155-414d-9384-52c9b1554f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.352541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e728e1-c155-414d-9384-52c9b1554f71 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.354082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 31/60] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:39.359666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67019e-fc93-42a1-967c-6f9317660283 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:39.453911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67019e-fc93-42a1-967c-6f9317660283 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:39.486091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07491936-4521-4a8f-9d8f-b3eb1fe4a3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:39.486698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07491936-4521-4a8f-9d8f-b3eb1fe4a3f8 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:39.486975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07491936-4521-4a8f-9d8f-b3eb1fe4a3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:39.487245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07491936-4521-4a8f-9d8f-b3eb1fe4a3f8 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:39.487882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07491936-4521-4a8f-9d8f-b3eb1fe4a3f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:39.489325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07491936-4521-4a8f-9d8f-b3eb1fe4a3f8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:39.572461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 30/61] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:39 2022] POST /identity/v3/users => generated 275 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:13:42.828358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e7898-044f-4763-84d9-5b0738b94470 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.829192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e7898-044f-4763-84d9-5b0738b94470 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.829639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e7898-044f-4763-84d9-5b0738b94470 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.831484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 32/62] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:42.837480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0ad9-63eb-45cb-98e1-341942e92ad8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.838253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0ad9-63eb-45cb-98e1-341942e92ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.838712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0ad9-63eb-45cb-98e1-341942e92ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.861297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-55af0ad9-63eb-45cb-98e1-341942e92ad8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:42.862803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-55af0ad9-63eb-45cb-98e1-341942e92ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:42.875040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459522a-3c6f-4a2c-b900-46a2d3a78222 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.875530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459522a-3c6f-4a2c-b900-46a2d3a78222 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.875941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459522a-3c6f-4a2c-b900-46a2d3a78222 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.877715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 33/63] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:42.883295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088fc17-c9e8-41d3-8b2d-b99d77149fb6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.883834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088fc17-c9e8-41d3-8b2d-b99d77149fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.884308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088fc17-c9e8-41d3-8b2d-b99d77149fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.909586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8088fc17-c9e8-41d3-8b2d-b99d77149fb6 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:42.912401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8088fc17-c9e8-41d3-8b2d-b99d77149fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:42.945416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:42.974500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 31/64] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:42.976555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebcf29-5b57-4df4-b6ca-331ad1304093 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.977324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebcf29-5b57-4df4-b6ca-331ad1304093 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.977869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebcf29-5b57-4df4-b6ca-331ad1304093 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:42.980459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 32/65] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:42.994953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101b3d-a896-417c-aa61-20bd211c8717 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:42.995520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101b3d-a896-417c-aa61-20bd211c8717 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:42.995983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101b3d-a896-417c-aa61-20bd211c8717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.013538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 34/66] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:43.014657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088fc17-c9e8-41d3-8b2d-b99d77149fb6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.019066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ed101b3d-a896-417c-aa61-20bd211c8717 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:43.021113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ed101b3d-a896-417c-aa61-20bd211c8717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:43.086204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.096923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 33/67] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:43.099591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed101b3d-a896-417c-aa61-20bd211c8717 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.139107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088fc17-c9e8-41d3-8b2d-b99d77149fb6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.176553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363313e2-56b6-429c-9cb0-86f7d0bd6c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.177374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363313e2-56b6-429c-9cb0-86f7d0bd6c6d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.177650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363313e2-56b6-429c-9cb0-86f7d0bd6c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.177919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363313e2-56b6-429c-9cb0-86f7d0bd6c6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.188742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363313e2-56b6-429c-9cb0-86f7d0bd6c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-363313e2-56b6-429c-9cb0-86f7d0bd6c6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:43.193918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:43.194989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:43.195790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 35/68] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:43.196545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183d43a-ec56-4538-bc4e-4d857aff4aef None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.196869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183d43a-ec56-4538-bc4e-4d857aff4aef None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.197147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183d43a-ec56-4538-bc4e-4d857aff4aef None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.199012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 36/69] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:43.203236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183d43a-ec56-4538-bc4e-4d857aff4aef None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.221416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed101b3d-a896-417c-aa61-20bd211c8717 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.236536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.258296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.258959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.259241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.259514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.270014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:43.275754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:43.276600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:43.277407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 34/70] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:43.278012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.311525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183d43a-ec56-4538-bc4e-4d857aff4aef None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.326840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.346996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51781f87-3ef0-4f90-a5aa-2624f11cf721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.347553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51781f87-3ef0-4f90-a5aa-2624f11cf721 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.347891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51781f87-3ef0-4f90-a5aa-2624f11cf721 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.348156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51781f87-3ef0-4f90-a5aa-2624f11cf721 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.358134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51781f87-3ef0-4f90-a5aa-2624f11cf721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-51781f87-3ef0-4f90-a5aa-2624f11cf721 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:43.363260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:43.364799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:43.365602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:43.366960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 37/71] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:43.368286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a056b8-43c6-4066-a3dd-04832fe8be2b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.368571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a056b8-43c6-4066-a3dd-04832fe8be2b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.368846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a056b8-43c6-4066-a3dd-04832fe8be2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.378963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988af5-fcb9-42ad-8e8b-8f4e0a4a18e3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.386618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-13a056b8-43c6-4066-a3dd-04832fe8be2b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:43.387796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-13a056b8-43c6-4066-a3dd-04832fe8be2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:43.415127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68118b3c-234a-411e-9efc-b48794cabafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.415726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118b3c-234a-411e-9efc-b48794cabafb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.416006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118b3c-234a-411e-9efc-b48794cabafb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.416282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118b3c-234a-411e-9efc-b48794cabafb None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.416940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118b3c-234a-411e-9efc-b48794cabafb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:43.417462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118b3c-234a-411e-9efc-b48794cabafb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.418665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118b3c-234a-411e-9efc-b48794cabafb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.427398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 35/72] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.428308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89666d92-da2e-4172-9f2c-8478887a0670 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.428591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89666d92-da2e-4172-9f2c-8478887a0670 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.428871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89666d92-da2e-4172-9f2c-8478887a0670 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.430246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 36/73] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:43.430805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89666d92-da2e-4172-9f2c-8478887a0670 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.455886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 38/74] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:43.462533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a056b8-43c6-4066-a3dd-04832fe8be2b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.474180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.539707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89666d92-da2e-4172-9f2c-8478887a0670 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.557399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a056b8-43c6-4066-a3dd-04832fe8be2b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.559687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.573371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.573974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.574257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.574562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.575226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:43.575530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.576952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.585399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 37/75] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.588946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.591655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbcb41-7a2b-4d2b-ab59-a77aa8d3fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.592262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbcb41-7a2b-4d2b-ab59-a77aa8d3fa5f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.592544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbcb41-7a2b-4d2b-ab59-a77aa8d3fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.592818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbcb41-7a2b-4d2b-ab59-a77aa8d3fa5f None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.593476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbcb41-7a2b-4d2b-ab59-a77aa8d3fa5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:43.593778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbcb41-7a2b-4d2b-ab59-a77aa8d3fa5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.595323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbcb41-7a2b-4d2b-ab59-a77aa8d3fa5f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.604199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 39/76] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.605291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb875c-235d-413f-bbe8-c475898cb3c5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.605580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb875c-235d-413f-bbe8-c475898cb3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.605857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb875c-235d-413f-bbe8-c475898cb3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.623541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-67fb875c-235d-413f-bbe8-c475898cb3c5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:43.624739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-67fb875c-235d-413f-bbe8-c475898cb3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:43.682377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.685812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e119199-961e-4595-bde3-068faf4074ee None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.691230 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 40/77] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:43.692061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb875c-235d-413f-bbe8-c475898cb3c5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.721221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.721922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.722279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.722679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] PATH_INFO: `/v3/users/c5565152ee164c0da536fef7f9cd3d6c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.724676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] RBAC: Authorizing `identity:get_user(user_id=c5565152ee164c0da536fef7f9cd3d6c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.726141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.727747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 38/78] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/users/c5565152ee164c0da536fef7f9cd3d6c => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.728500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.787378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.793157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb875c-235d-413f-bbe8-c475898cb3c5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.827331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.827728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14db93b-b6f5-4559-8764-47a95c9c1053 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.828102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.828443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.828963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.829883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.831376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.832986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 41/79] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.838054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.866619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.867185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.867469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.868008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.881142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.882638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.884249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 39/80] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.885013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.909727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:43.939305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb80cc1-c31d-4217-ac3f-baf74d3e9e95 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:43.972600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:43.973216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:43.974063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:43.974393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:43.975543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:43.976974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:43.978557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 42/81] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:43.983270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:43.991316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88860d7-0c33-49da-89e8-7f3be1f15102 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.009745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.025214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.025867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.026146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.026449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.037176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:44.041894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:44.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:44.043705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 40/82] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:44.044324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.082369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acae2ad-6278-4562-9daa-1fb5cc68f7d8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.116168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.116847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.117072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.117341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.126872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.142003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787dcda-ea53-487b-a4f6-df554e4eda45 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.176329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.176950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.177233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.177511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.178150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.179628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.184203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.185640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.215012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.280747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 41/83] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:44 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:44.281237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 43/84] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:44.281863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.282458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.399468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b409765-d984-486d-aeb6-f09a742e4956 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.405946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e1f96-14a3-49a2-95e1-48691e3710d7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.432891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.439253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.439784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.440068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.440347 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.444126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.444978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.445225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.445495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.457732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.525728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.528190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.530451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.531927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.643474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 42/85] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:44.644665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.649877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 44/86] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:44.650689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.684221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.691359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.757329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8fb5e-3e46-4752-abf7-727f1f557d10 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.759727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2604367-eafe-4d27-a2df-ac0c850e7262 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.796326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.796707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.797340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.797625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.797903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.798023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.798606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.799067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.799246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.800104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:44.800505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.800814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.803091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 45/87] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:44.803625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.803796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.813765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 43/88] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:44.815017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.909822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.913549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa8e3f-80ee-429e-93c7-33fab219b74f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.937556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88945d5-0758-4584-a80d-ce3e38087bef None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:44.940284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:44.954418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.955428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.955871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.956303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.961118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.961118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.962105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 46/89] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:44.963092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:44.978982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:44.980057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:44.980557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:44.981074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:44.982418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:44.984347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:44.986183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 44/90] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:44.987250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.078412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d18ca-8ba5-4523-b8a6-4f6ba372d897 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.110838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaeef4f-7daf-4af5-8bf7-c3a5784cf890 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.120437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.121462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.121943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.122620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.123594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=c5565152ee164c0da536fef7f9cd3d6c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.124145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.126041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.136004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 47/91] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/role_assignments?user.id=c5565152ee164c0da536fef7f9cd3d6c&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.137211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.163851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.164415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.164693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.164964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.167835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.169411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.171159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 45/92] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:44 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.172867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.176014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.260431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.307629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3886a-2043-42ad-9ebb-ae9ee326efe1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.371653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.372936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.373675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.374595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.375814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] RBAC: Adding query filter params (role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=82f38e27cd95460f986105f4aeedb6ef) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.376344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.377915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b49d9-23a6-42c7-9327-a11569976e33 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.379132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.387674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 48/93] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/role_assignments?user.id=82f38e27cd95460f986105f4aeedb6ef&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.388921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:45.449766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.450305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.450630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.450902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.451638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=82f38e27cd95460f986105f4aeedb6ef) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:45.451931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.453483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.472126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 46/94] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/role_assignments?user.id=82f38e27cd95460f986105f4aeedb6ef&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:45.558809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.565324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d65990-8df8-4e6d-929c-3f8d9e2e8036 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:45.618744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:45.619380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:45.619661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:45.619933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:45.716613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:45.718018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:45.876944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:45.885486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 49/95] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:45.891094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:46.033358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62739-fa9c-4517-a90a-fc804bfe1a4e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:46.035616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:46.070301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987003c4-f555-444e-8412-1ef899d62942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:46.070960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987003c4-f555-444e-8412-1ef899d62942 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:46.071241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987003c4-f555-444e-8412-1ef899d62942 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:46.071514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987003c4-f555-444e-8412-1ef899d62942 None admin] PATH_INFO: `/v3/projects/44def63210f9490082cf44969ebd9971` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:46.072664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987003c4-f555-444e-8412-1ef899d62942 None admin] RBAC: Authorizing `identity:get_project(project_id=44def63210f9490082cf44969ebd9971)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:46.074372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987003c4-f555-444e-8412-1ef899d62942 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:46.076030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 47/96] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:45 2022] GET /identity/v3/projects/44def63210f9490082cf44969ebd9971 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:46.081990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:46.215238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ddad8-a453-4b06-a4a5-8239e4226fcd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:46.250129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38343587-6b61-42b6-ab31-02b48ff92f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:46.250769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38343587-6b61-42b6-ab31-02b48ff92f79 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:46.251052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38343587-6b61-42b6-ab31-02b48ff92f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:46.251402 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38343587-6b61-42b6-ab31-02b48ff92f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:46.252232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38343587-6b61-42b6-ab31-02b48ff92f79 None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=44def63210f9490082cf44969ebd9971, user.id=82f38e27cd95460f986105f4aeedb6ef) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:46.260560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38343587-6b61-42b6-ab31-02b48ff92f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:46.260560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38343587-6b61-42b6-ab31-02b48ff92f79 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:46.260560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 50/97] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:46 2022] GET /identity/v3/role_assignments?user.id=82f38e27cd95460f986105f4aeedb6ef&scope.project.id=44def63210f9490082cf44969ebd9971&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:47.091660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f607c-08af-4947-b5d2-0919f7e4b8d2 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.092184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f607c-08af-4947-b5d2-0919f7e4b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.092184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f607c-08af-4947-b5d2-0919f7e4b8d2 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.093810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 48/98] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:47.099469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27a76-04d6-419c-b4e5-0ec41b0207aa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.099870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27a76-04d6-419c-b4e5-0ec41b0207aa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.100136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27a76-04d6-419c-b4e5-0ec41b0207aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.127329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-49f27a76-04d6-419c-b4e5-0ec41b0207aa None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:47.128534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-49f27a76-04d6-419c-b4e5-0ec41b0207aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:47.157586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935deb6-fb3b-47e7-b658-b6d7fe01644b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.157890 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935deb6-fb3b-47e7-b658-b6d7fe01644b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.158144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935deb6-fb3b-47e7-b658-b6d7fe01644b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.159667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 49/99] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:13:47.165539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaffaa7-eeb1-4aa6-9648-355cedc868a3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.165823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaffaa7-eeb1-4aa6-9648-355cedc868a3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.166096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaffaa7-eeb1-4aa6-9648-355cedc868a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.186106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-baaffaa7-eeb1-4aa6-9648-355cedc868a3 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:47.187819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-baaffaa7-eeb1-4aa6-9648-355cedc868a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:47.196506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:47.228681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 51/100] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:47.266415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 50/101] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:47.271572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f27a76-04d6-419c-b4e5-0ec41b0207aa None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:47.320547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaffaa7-eeb1-4aa6-9648-355cedc868a3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:47.386326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f27a76-04d6-419c-b4e5-0ec41b0207aa None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:47.424871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:47.425506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.426034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.426310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.436354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:47.437675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:47.439362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 52/102] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:47.445777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:47.574725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:47.650814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:47.658652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaffaa7-eeb1-4aa6-9648-355cedc868a3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:47.688771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029946-fc04-420c-8944-c4a2fedeae80 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:47.753978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:47.755169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.755450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.755722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.771767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e7d22-b116-4aef-ad84-315dc06d2f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:47.772667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7d22-b116-4aef-ad84-315dc06d2f8e None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.773152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7d22-b116-4aef-ad84-315dc06d2f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.773620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7d22-b116-4aef-ad84-315dc06d2f8e None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.774494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e7d22-b116-4aef-ad84-315dc06d2f8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:47.776107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e7d22-b116-4aef-ad84-315dc06d2f8e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:47.783274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:47.792781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:47.793612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:47.794862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 51/103] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 474 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:47.803312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:47.897645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 53/104] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:13:47 2022] POST /identity/v3/projects => generated 302 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:13:47.899718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b462af6-96ac-4da1-82e3-cfb65946bded None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.900863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b462af6-96ac-4da1-82e3-cfb65946bded None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.901564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b462af6-96ac-4da1-82e3-cfb65946bded None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.904630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 54/105] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:47.910703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bc744-47ee-4a87-9eab-3f1bd9f9a182 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:47.911516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bc744-47ee-4a87-9eab-3f1bd9f9a182 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:47.911845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bc744-47ee-4a87-9eab-3f1bd9f9a182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:47.955626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-227bc744-47ee-4a87-9eab-3f1bd9f9a182 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:47.956813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-227bc744-47ee-4a87-9eab-3f1bd9f9a182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:47.998779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e59fd2-2cae-4860-b1b4-f81df4efe3e0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.071146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.072124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.072624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.073189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.074085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:48.074605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.076433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.093030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 52/106] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:48.099049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.114997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 55/107] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:48.133208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.146737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bc744-47ee-4a87-9eab-3f1bd9f9a182 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.207184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.279357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbdc94-ef27-4cb1-add7-7a68d1497724 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.306390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bc744-47ee-4a87-9eab-3f1bd9f9a182 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.352507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e107fb-36cf-4af8-97b3-4b8b887aa7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.353244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e107fb-36cf-4af8-97b3-4b8b887aa7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.358431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e107fb-36cf-4af8-97b3-4b8b887aa7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.358431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e107fb-36cf-4af8-97b3-4b8b887aa7a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.368856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e107fb-36cf-4af8-97b3-4b8b887aa7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-19e107fb-36cf-4af8-97b3-4b8b887aa7a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:48.373643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:48.374693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:48.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:48.377421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 56/108] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:48.379219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a647f-eb8e-4678-95ca-a99b46d8ddc0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.380253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a647f-eb8e-4678-95ca-a99b46d8ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.380816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a647f-eb8e-4678-95ca-a99b46d8ddc0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.382326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 57/109] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:48.387168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a647f-eb8e-4678-95ca-a99b46d8ddc0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.389655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096ed8-3fd1-4925-8152-3e0c1d9a513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.390806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096ed8-3fd1-4925-8152-3e0c1d9a513e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.392215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096ed8-3fd1-4925-8152-3e0c1d9a513e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.392642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096ed8-3fd1-4925-8152-3e0c1d9a513e None admin] PATH_INFO: `/v3/users/35820b183a584ac787f45a974438a379` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.412746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096ed8-3fd1-4925-8152-3e0c1d9a513e None admin] RBAC: Authorizing `identity:get_user(user_id=35820b183a584ac787f45a974438a379)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.414957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096ed8-3fd1-4925-8152-3e0c1d9a513e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.416985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 53/110] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/users/35820b183a584ac787f45a974438a379 => generated 289 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:48.418547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55408c9-b8e8-4109-9993-90c9320de84d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.419323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55408c9-b8e8-4109-9993-90c9320de84d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.419773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55408c9-b8e8-4109-9993-90c9320de84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.440304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a55408c9-b8e8-4109-9993-90c9320de84d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:48.442711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a55408c9-b8e8-4109-9993-90c9320de84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:48.565319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.569982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a647f-eb8e-4678-95ca-a99b46d8ddc0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.570700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 54/111] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:48.571573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55408c9-b8e8-4109-9993-90c9320de84d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.596396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.630470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.631468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.632159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.633113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.634177 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:48.634701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.636319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.654260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 58/112] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:48.655586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.750829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55408c9-b8e8-4109-9993-90c9320de84d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.789059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2a05f-6a18-469c-a92e-7b9eef7c9e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.789668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2a05f-6a18-469c-a92e-7b9eef7c9e3e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.789947 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2a05f-6a18-469c-a92e-7b9eef7c9e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.790215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2a05f-6a18-469c-a92e-7b9eef7c9e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.791814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214a924-b0f4-4f0f-8f75-6cc490cc3bea None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.830498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:48.831479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.831968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.832424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.845156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:48.852193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:48.853033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:48.853791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:48.856090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 59/113] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:48.858073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:48.863650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2a05f-6a18-469c-a92e-7b9eef7c9e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:48.865453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2a05f-6a18-469c-a92e-7b9eef7c9e3e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:48.879066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.888698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:48.970478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fc176-dddf-4f95-8e00-be4d40e10c96 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:48.972570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 55/114] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:48.974675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2ae2-8d69-44d3-b32e-16fb13677ce3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:48.975064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2ae2-8d69-44d3-b32e-16fb13677ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:48.975445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2ae2-8d69-44d3-b32e-16fb13677ce3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:48.976804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 56/115] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:48.977587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf2ae2-8d69-44d3-b32e-16fb13677ce3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.011642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad293a-cf70-40bc-8a3c-b97e97f2f87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.012542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad293a-cf70-40bc-8a3c-b97e97f2f87b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.012957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad293a-cf70-40bc-8a3c-b97e97f2f87b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.013362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad293a-cf70-40bc-8a3c-b97e97f2f87b None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.014686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad293a-cf70-40bc-8a3c-b97e97f2f87b None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.016256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad293a-cf70-40bc-8a3c-b97e97f2f87b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.018091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 60/116] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.019536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6b5c0-8e7f-4e7e-a321-1011f4944078 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.019953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6b5c0-8e7f-4e7e-a321-1011f4944078 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.020990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6b5c0-8e7f-4e7e-a321-1011f4944078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.039891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-32c6b5c0-8e7f-4e7e-a321-1011f4944078 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:49.041310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-32c6b5c0-8e7f-4e7e-a321-1011f4944078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:49.089598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf2ae2-8d69-44d3-b32e-16fb13677ce3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.101246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.113711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 61/117] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:49.115481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6b5c0-8e7f-4e7e-a321-1011f4944078 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.127522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.128205 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.128567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.128901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.130224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:49.130683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.132128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.140592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.142602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 57/118] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.143615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.233540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6b5c0-8e7f-4e7e-a321-1011f4944078 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.254144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8eb2d0-8cad-41c5-9358-3f8ef6a2ca9c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.270192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.271175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.271479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.271729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.272866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.274644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.276641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 62/119] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.277328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.293366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.294034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.294340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.294618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.295770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.297250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.299197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 58/120] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.299768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.349367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.375553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.387504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a9b43-6bbb-4364-891e-63a783a1ddb4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.419362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0404b-b54b-4783-9cd8-02cf1793e532 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.422740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea551cbf-a794-4c65-b5c8-8bddab1842d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.423393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea551cbf-a794-4c65-b5c8-8bddab1842d8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.423669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea551cbf-a794-4c65-b5c8-8bddab1842d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.423938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea551cbf-a794-4c65-b5c8-8bddab1842d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.436174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea551cbf-a794-4c65-b5c8-8bddab1842d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ea551cbf-a794-4c65-b5c8-8bddab1842d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:49.440902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:49.441721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:49.442780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 63/121] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:49.443396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309bba-1996-4e96-92ba-2c7b60e11d17 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.443591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309bba-1996-4e96-92ba-2c7b60e11d17 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.443773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309bba-1996-4e96-92ba-2c7b60e11d17 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.446609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 64/122] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:49.447307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309bba-1996-4e96-92ba-2c7b60e11d17 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.469123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.469762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.470050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.470329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] PATH_INFO: `/v3/users/c5565152ee164c0da536fef7f9cd3d6c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.471503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] RBAC: Authorizing `identity:get_user(user_id=c5565152ee164c0da536fef7f9cd3d6c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.472990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.475387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 59/123] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/users/c5565152ee164c0da536fef7f9cd3d6c => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.475978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.562117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309bba-1996-4e96-92ba-2c7b60e11d17 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.575938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00742c-2390-4bcd-9079-1a78fe3533c4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.585309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.601485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db70d97-0fe0-4e57-ad96-5424f60d5d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.601485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db70d97-0fe0-4e57-ad96-5424f60d5d80 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.601485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db70d97-0fe0-4e57-ad96-5424f60d5d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.601485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db70d97-0fe0-4e57-ad96-5424f60d5d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.601828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db70d97-0fe0-4e57-ad96-5424f60d5d80 None admin] RBAC: Adding query filter params (role.id=ffd19d9da2c74b20b0831c13c655ec94, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=35820b183a584ac787f45a974438a379) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:49.601828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db70d97-0fe0-4e57-ad96-5424f60d5d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.603088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db70d97-0fe0-4e57-ad96-5424f60d5d80 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.609057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 65/124] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/role_assignments?user.id=35820b183a584ac787f45a974438a379&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=ffd19d9da2c74b20b0831c13c655ec94 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.610034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcf38e-5373-48ec-8cc6-599c00007a04 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.610323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcf38e-5373-48ec-8cc6-599c00007a04 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.610626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcf38e-5373-48ec-8cc6-599c00007a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.625785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.627294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.627837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.628339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/82f38e27cd95460f986105f4aeedb6ef/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.631836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf931c3358334a969b8a22fa1edbbe55, user_id=82f38e27cd95460f986105f4aeedb6ef, role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.632012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bdfcf38e-5373-48ec-8cc6-599c00007a04 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:49.633280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bdfcf38e-5373-48ec-8cc6-599c00007a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:49.635309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.643548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.702062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 60/125] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:49 2022] PUT /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/82f38e27cd95460f986105f4aeedb6ef/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:49.704266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.719738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 66/126] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:49.720679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcf38e-5373-48ec-8cc6-599c00007a04 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.849706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0827c-99e1-4ef1-aa29-9ed22dc3e784 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.920044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.921051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.922744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.923213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.925445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:49.927445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.930991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.939465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcf38e-5373-48ec-8cc6-599c00007a04 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:49.942743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 61/127] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.945126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:49.983075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:49.983916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:49.984296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:49.984665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:49.985294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:49.986404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:49.987990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:49.993668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 67/128] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:49.994657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.008881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.081673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb8276-5a56-41b9-9a73-fb9bee252da1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.125305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510db4-731f-47ec-80d8-7b6a2d08ffea None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.166124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.167073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.167913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.168483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.169839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.171600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.173390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 68/129] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.174451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.195678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.196296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.196585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.196860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.217054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:50.228650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:50.229797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 62/130] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:50.230666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.271143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.298756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45a377-81a7-4a4e-b47a-444b4cd62ef3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.340025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.341100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.341613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.342016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/c5565152ee164c0da536fef7f9cd3d6c/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.344675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] RBAC: Authorizing `identity:create_grant(project_id=cf931c3358334a969b8a22fa1edbbe55, user_id=c5565152ee164c0da536fef7f9cd3d6c, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.346321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.403847 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.408626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 69/131] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:50 2022] PUT /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/c5565152ee164c0da536fef7f9cd3d6c/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:50.409729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.416454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a5645-e921-4389-8de2-4c1f88d4bdf4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.464773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0401-4ec3-4440-8fd0-a70bf8d656bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.465338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0401-4ec3-4440-8fd0-a70bf8d656bf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.465620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0401-4ec3-4440-8fd0-a70bf8d656bf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.465896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0401-4ec3-4440-8fd0-a70bf8d656bf None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.467208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0401-4ec3-4440-8fd0-a70bf8d656bf None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.469475 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0401-4ec3-4440-8fd0-a70bf8d656bf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.471960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 63/132] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.473926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51566a01-beca-4d84-8c6c-38b4f440b93b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.474219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51566a01-beca-4d84-8c6c-38b4f440b93b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.474510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51566a01-beca-4d84-8c6c-38b4f440b93b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.478436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 64/133] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:50.479587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51566a01-beca-4d84-8c6c-38b4f440b93b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.554407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392bd7ea-8bf9-433a-a190-89c78f49f72d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.601321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e10997-d86d-4e3e-afea-7fa117655eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.601947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e10997-d86d-4e3e-afea-7fa117655eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.602227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e10997-d86d-4e3e-afea-7fa117655eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.602539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e10997-d86d-4e3e-afea-7fa117655eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.603222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e10997-d86d-4e3e-afea-7fa117655eb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:50.603524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e10997-d86d-4e3e-afea-7fa117655eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.604935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e10997-d86d-4e3e-afea-7fa117655eb5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.623486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.632734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 70/134] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.634248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52fc37-422c-4fd5-b498-cca5f2dc24df None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.635168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52fc37-422c-4fd5-b498-cca5f2dc24df None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.635168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52fc37-422c-4fd5-b498-cca5f2dc24df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.657183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51566a01-beca-4d84-8c6c-38b4f440b93b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.664347 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9a52fc37-422c-4fd5-b498-cca5f2dc24df None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:50.665702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9a52fc37-422c-4fd5-b498-cca5f2dc24df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:50.697628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.698293 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.698611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.698889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/82f38e27cd95460f986105f4aeedb6ef/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.701000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf931c3358334a969b8a22fa1edbbe55, user_id=82f38e27cd95460f986105f4aeedb6ef, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.702523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.741863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 71/135] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:50.742789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52fc37-422c-4fd5-b498-cca5f2dc24df None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.777666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 65/136] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:50 2022] PUT /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/82f38e27cd95460f986105f4aeedb6ef/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:50.782863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:50.823760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.853617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52fc37-422c-4fd5-b498-cca5f2dc24df None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:50.885738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:50.892066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:50.892742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:50.893025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:50.893304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:50.894487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:50.896482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:50.898261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 72/137] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:50.903670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.001618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b933ef6-81b7-4845-ae97-7ddacce7a552 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.078220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.078901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.079184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.079452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.080901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.082395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.084532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 66/138] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:51.092974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.110719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6408bb-0e41-4d10-a36e-6d757dfa3bf7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.153668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.154436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.154652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.154931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] PATH_INFO: `/v3/projects/44def63210f9490082cf44969ebd9971` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.159360 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] RBAC: Authorizing `identity:get_project(project_id=44def63210f9490082cf44969ebd9971)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.161442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.166315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 73/139] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:50 2022] GET /identity/v3/projects/44def63210f9490082cf44969ebd9971 => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:51.173406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:51.271053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.302934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdcd9a-0ffd-4733-ab39-8da073dfcd56 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.306714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:51.385888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f543e-65e0-4c08-8c73-bcd8d78615fb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:51.387039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef286d7-6af4-4fd2-a644-3d31c6704c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.387823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef286d7-6af4-4fd2-a644-3d31c6704c20 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.388105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef286d7-6af4-4fd2-a644-3d31c6704c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.388378 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef286d7-6af4-4fd2-a644-3d31c6704c20 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.389029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef286d7-6af4-4fd2-a644-3d31c6704c20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.390591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef286d7-6af4-4fd2-a644-3d31c6704c20 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.453250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438952fe-6307-49ac-8762-004d88a5fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:51.454035 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438952fe-6307-49ac-8762-004d88a5fd34 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:51.454331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438952fe-6307-49ac-8762-004d88a5fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:51.454630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438952fe-6307-49ac-8762-004d88a5fd34 None admin] PATH_INFO: `/v3/projects/44def63210f9490082cf44969ebd9971/users/82f38e27cd95460f986105f4aeedb6ef/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:51.463196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438952fe-6307-49ac-8762-004d88a5fd34 None admin] RBAC: Authorizing `identity:create_grant(project_id=44def63210f9490082cf44969ebd9971, user_id=82f38e27cd95460f986105f4aeedb6ef, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:51.464778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438952fe-6307-49ac-8762-004d88a5fd34 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:51.473571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 67/140] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:51 2022] POST /identity/v3/users => generated 283 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:51.546085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 74/141] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:51 2022] PUT /identity/v3/projects/44def63210f9490082cf44969ebd9971/users/82f38e27cd95460f986105f4aeedb6ef/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:52.313102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c319c-1bdf-4b05-9f95-8fae95db3726 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.314137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c319c-1bdf-4b05-9f95-8fae95db3726 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.314679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c319c-1bdf-4b05-9f95-8fae95db3726 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.316496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 68/142] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:52.322484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8b2ee-eaed-43e0-aedf-e6bb01728997 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.323386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8b2ee-eaed-43e0-aedf-e6bb01728997 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.323868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8b2ee-eaed-43e0-aedf-e6bb01728997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.345474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d7a8b2ee-eaed-43e0-aedf-e6bb01728997 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:52.347068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d7a8b2ee-eaed-43e0-aedf-e6bb01728997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:52.421810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 75/143] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:52.466371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c319c-1bdf-4b05-9f95-8fae95db3726 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.584910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c319c-1bdf-4b05-9f95-8fae95db3726 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.621861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee00d0-0124-4aa2-82f4-639107a64c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.622928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee00d0-0124-4aa2-82f4-639107a64c0a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.623479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee00d0-0124-4aa2-82f4-639107a64c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.623950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee00d0-0124-4aa2-82f4-639107a64c0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.635897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee00d0-0124-4aa2-82f4-639107a64c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6cee00d0-0124-4aa2-82f4-639107a64c0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:52.641530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:52.642346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:52.643286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:52.645780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 69/144] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:52.653718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8b2ee-eaed-43e0-aedf-e6bb01728997 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.714595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:52.766888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8b2ee-eaed-43e0-aedf-e6bb01728997 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:52.799402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdae2e-b2e9-4302-8187-0522fa134fbe None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.799695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdae2e-b2e9-4302-8187-0522fa134fbe None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.799975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdae2e-b2e9-4302-8187-0522fa134fbe None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.801926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 70/145] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:52.810297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b7785-c257-4a74-b269-65b82b232639 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.810607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b7785-c257-4a74-b269-65b82b232639 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.810889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b7785-c257-4a74-b269-65b82b232639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.817867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:52.832729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:52.833534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:52.833820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:52.837555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:52.837555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:52.837555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:52.838614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:52.840843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-169b7785-c257-4a74-b269-65b82b232639 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:52.842128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-169b7785-c257-4a74-b269-65b82b232639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:52.860039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 76/146] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:52.866764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:52.990253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 71/147] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:13:53.022607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b7785-c257-4a74-b269-65b82b232639 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.066406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bf331-5ee7-491c-96f6-c77a5ac27997 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.107526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.151092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.151913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.152303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.152582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.153795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.155337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.156983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 77/148] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:52 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:53.165248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.258705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b7785-c257-4a74-b269-65b82b232639 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.328206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.341099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d9810-fd2f-4973-98a1-cd403e94ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.341736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d9810-fd2f-4973-98a1-cd403e94ab29 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.342008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d9810-fd2f-4973-98a1-cd403e94ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.342274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d9810-fd2f-4973-98a1-cd403e94ab29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.364136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d9810-fd2f-4973-98a1-cd403e94ab29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c93d9810-fd2f-4973-98a1-cd403e94ab29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:53.369195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:53.370059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:53.370939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 72/149] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:13:53.372066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934aca2-b667-4e29-8e8c-e4c208232c19 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.372869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934aca2-b667-4e29-8e8c-e4c208232c19 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.372869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934aca2-b667-4e29-8e8c-e4c208232c19 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.373728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 73/150] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:53.383308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180cdd4-9e1b-4791-a1d9-dcdcdd01ba77 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.391073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65f49-57b0-4334-96c2-bbf7294f5514 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.391409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65f49-57b0-4334-96c2-bbf7294f5514 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.391621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b65f49-57b0-4334-96c2-bbf7294f5514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.414898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c9b65f49-57b0-4334-96c2-bbf7294f5514 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:53.416076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c9b65f49-57b0-4334-96c2-bbf7294f5514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:53.444029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.444658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.444999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.445275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.520725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 74/151] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:53.521635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b65f49-57b0-4334-96c2-bbf7294f5514 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.568487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.598715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.600205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.694741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b65f49-57b0-4334-96c2-bbf7294f5514 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.730999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.763085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d25d44-0f40-46bf-9c8b-ec0f017140ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:53.763712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d25d44-0f40-46bf-9c8b-ec0f017140ff None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.763994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d25d44-0f40-46bf-9c8b-ec0f017140ff None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.764265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d25d44-0f40-46bf-9c8b-ec0f017140ff None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.764592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 78/152] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:13:53.765021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d25d44-0f40-46bf-9c8b-ec0f017140ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:53.765329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d25d44-0f40-46bf-9c8b-ec0f017140ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:53.765486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.766849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d25d44-0f40-46bf-9c8b-ec0f017140ff None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:53.781663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 75/153] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:53.783001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86494221-5c31-4f14-bca2-f8800b95ee63 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:53.783295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86494221-5c31-4f14-bca2-f8800b95ee63 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:53.783564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86494221-5c31-4f14-bca2-f8800b95ee63 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:53.784911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 76/154] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:53.789549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86494221-5c31-4f14-bca2-f8800b95ee63 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:53.933647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:53.958386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898cb80-3a78-4125-97de-525a3dab22b8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:53.975383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86494221-5c31-4f14-bca2-f8800b95ee63 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.027173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.027804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.028081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.028351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.037087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a99bf3-6441-4544-89bb-3d6d0a921962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.037711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99bf3-6441-4544-89bb-3d6d0a921962 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.037993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99bf3-6441-4544-89bb-3d6d0a921962 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.038257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99bf3-6441-4544-89bb-3d6d0a921962 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.039534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99bf3-6441-4544-89bb-3d6d0a921962 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.040983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99bf3-6441-4544-89bb-3d6d0a921962 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.042644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 77/155] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:54.045024 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf1f4-b387-43d6-bd8a-4c4db76b2853 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.045305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf1f4-b387-43d6-bd8a-4c4db76b2853 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.045569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf1f4-b387-43d6-bd8a-4c4db76b2853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.052294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:54.057001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:54.057931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:54.058745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 79/156] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:54.059416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.089654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6dbdf1f4-b387-43d6-bd8a-4c4db76b2853 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:54.092845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6dbdf1f4-b387-43d6-bd8a-4c4db76b2853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:54.215523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.253552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 78/157] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:54.255300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdf1f4-b387-43d6-bd8a-4c4db76b2853 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.310131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca579f9f-3bd8-47f9-a7fd-745a31c55fbd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.395612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.396431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.396511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.396783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] PATH_INFO: `/v3/users/35820b183a584ac787f45a974438a379` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.397986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] RBAC: Authorizing `identity:get_user(user_id=35820b183a584ac787f45a974438a379)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.402374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.402374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 80/158] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/users/35820b183a584ac787f45a974438a379 => generated 289 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:54.402374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.435640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.460568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdf1f4-b387-43d6-bd8a-4c4db76b2853 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.527033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.528001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.528474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.528934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.529856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] RBAC: Adding query filter params (role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=82f38e27cd95460f986105f4aeedb6ef) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:54.530399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.532114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.539359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 79/159] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/role_assignments?user.id=82f38e27cd95460f986105f4aeedb6ef&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:54.540453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.633967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:54.644550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abfe24-28bf-4af5-8b10-a6aeecc65a62 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.727671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d68973-9fd6-4989-a07a-ec94ab319764 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:54.755650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.756287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.756571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.756838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.757524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:54.757826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:54.759278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:54.772473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 81/160] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:54.773268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.792882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:54.793532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:54.793810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:54.794080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:54.823147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:54.828411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:54.829275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:54.830034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 80/161] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:54.834833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:54.979840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.005447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58057b5f-24e9-4897-9f94-80c43de17372 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.030720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.033900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5000295-6474-4909-89cf-b7562e380707 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.055375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.056329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.056807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.057276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] PATH_INFO: `/v3/users/c5565152ee164c0da536fef7f9cd3d6c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.058798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] RBAC: Authorizing `identity:get_user(user_id=c5565152ee164c0da536fef7f9cd3d6c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.060460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.063745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 81/162] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/users/c5565152ee164c0da536fef7f9cd3d6c => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.066402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.104846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5c4b5-6a4f-45b5-850e-554e85591385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.106054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c4b5-6a4f-45b5-850e-554e85591385 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.110546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c4b5-6a4f-45b5-850e-554e85591385 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.111034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5c4b5-6a4f-45b5-850e-554e85591385 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.112406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5c4b5-6a4f-45b5-850e-554e85591385 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.116760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5c4b5-6a4f-45b5-850e-554e85591385 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.120164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 82/163] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:54 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.122604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7769b36-5cb3-469f-a006-03f72d65d488 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.125218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7769b36-5cb3-469f-a006-03f72d65d488 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.125699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7769b36-5cb3-469f-a006-03f72d65d488 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.128209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 83/164] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:55.130856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7769b36-5cb3-469f-a006-03f72d65d488 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.294087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d31d7-7eec-4b7f-90a8-c4fba42496fe None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.309523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7769b36-5cb3-469f-a006-03f72d65d488 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.362559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.364099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.364781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.365272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.366317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:55.366913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.368763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.377643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 82/165] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.386386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a2927-60dd-47c6-b292-1a5d22ccb655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.386386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a2927-60dd-47c6-b292-1a5d22ccb655 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.386386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a2927-60dd-47c6-b292-1a5d22ccb655 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.386386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a2927-60dd-47c6-b292-1a5d22ccb655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.387273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.399356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.430730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.496445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a2927-60dd-47c6-b292-1a5d22ccb655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.497933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a2927-60dd-47c6-b292-1a5d22ccb655 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.500357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c79fe-9254-419c-96f9-53dea6a61d61 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.577272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.578402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.578883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.579451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/35820b183a584ac787f45a974438a379/roles/ffd19d9da2c74b20b0831c13c655ec94` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.587254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] RBAC: Authorizing `identity:create_grant(project_id=cf931c3358334a969b8a22fa1edbbe55, user_id=35820b183a584ac787f45a974438a379, role_id=ffd19d9da2c74b20b0831c13c655ec94)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.588972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.603425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 84/166] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:55.604951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8768-2f38-4944-9d24-1e5ad7fe7c7d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.605468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8768-2f38-4944-9d24-1e5ad7fe7c7d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.605885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f8768-2f38-4944-9d24-1e5ad7fe7c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.625120 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c22f8768-2f38-4944-9d24-1e5ad7fe7c7d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:55.626315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c22f8768-2f38-4944-9d24-1e5ad7fe7c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:55.668646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 83/167] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:13:55 2022] PUT /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55/users/35820b183a584ac787f45a974438a379/roles/ffd19d9da2c74b20b0831c13c655ec94 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:13:55.669298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.669488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.699073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 85/168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:55.699959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8768-2f38-4944-9d24-1e5ad7fe7c7d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.811130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f8768-2f38-4944-9d24-1e5ad7fe7c7d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.811922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.826085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaabb9-ae8a-40c1-a1e4-925417dba8cd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:55.850770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c1dd1-9a57-4125-9ce9-1037f744ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.851325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c1dd1-9a57-4125-9ce9-1037f744ebec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.851606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c1dd1-9a57-4125-9ce9-1037f744ebec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.851882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c1dd1-9a57-4125-9ce9-1037f744ebec None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.853039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c1dd1-9a57-4125-9ce9-1037f744ebec None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.856006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c1dd1-9a57-4125-9ce9-1037f744ebec None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.856769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 86/169] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.858004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af6533-5c8e-4e5b-ba0b-2041e07f28c4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.858290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af6533-5c8e-4e5b-ba0b-2041e07f28c4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.858604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af6533-5c8e-4e5b-ba0b-2041e07f28c4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.859812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 87/170] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:55.860512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af6533-5c8e-4e5b-ba0b-2041e07f28c4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:55.885544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501196ea-b390-4377-bb25-d4eb65ea50b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:55.886192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501196ea-b390-4377-bb25-d4eb65ea50b2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.886546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501196ea-b390-4377-bb25-d4eb65ea50b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.886825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501196ea-b390-4377-bb25-d4eb65ea50b2 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.888107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501196ea-b390-4377-bb25-d4eb65ea50b2 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:55.889724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501196ea-b390-4377-bb25-d4eb65ea50b2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:55.891536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:55.892788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592ca0-75cd-4735-a6c1-1a7d115489b1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:55.893076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592ca0-75cd-4735-a6c1-1a7d115489b1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:55.893350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592ca0-75cd-4735-a6c1-1a7d115489b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:55.922504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:55.927423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ac592ca0-75cd-4735-a6c1-1a7d115489b1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:55.928667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ac592ca0-75cd-4735-a6c1-1a7d115489b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:55.981494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af6533-5c8e-4e5b-ba0b-2041e07f28c4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.019327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 85/172] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:56.021207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592ca0-75cd-4735-a6c1-1a7d115489b1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.031060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.031716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.031998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.032270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.061300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:56.066037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:56.066958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:56.067816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 88/173] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:56.068452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.147015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.154337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592ca0-75cd-4735-a6c1-1a7d115489b1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.187026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e2496-5136-42ab-96b7-329516fffa59 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.188570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.193013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e356a-9265-4fff-8807-9597d89000a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.193616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e356a-9265-4fff-8807-9597d89000a3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.193891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e356a-9265-4fff-8807-9597d89000a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.194158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e356a-9265-4fff-8807-9597d89000a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.194914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e356a-9265-4fff-8807-9597d89000a3 None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=c5565152ee164c0da536fef7f9cd3d6c) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:56.195338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e356a-9265-4fff-8807-9597d89000a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.198595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e356a-9265-4fff-8807-9597d89000a3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.202811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 86/174] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/role_assignments?user.id=c5565152ee164c0da536fef7f9cd3d6c&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.203958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323349ec-5747-4851-9e96-dce4430bfdda None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.204239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323349ec-5747-4851-9e96-dce4430bfdda None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.204508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323349ec-5747-4851-9e96-dce4430bfdda None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.205730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 87/175] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:56.206439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323349ec-5747-4851-9e96-dce4430bfdda None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.233673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84bf4a-9940-4876-9796-d64aeb33aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.234732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84bf4a-9940-4876-9796-d64aeb33aa70 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.235185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84bf4a-9940-4876-9796-d64aeb33aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.235626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84bf4a-9940-4876-9796-d64aeb33aa70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.325936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84bf4a-9940-4876-9796-d64aeb33aa70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.327512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84bf4a-9940-4876-9796-d64aeb33aa70 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.335497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323349ec-5747-4851-9e96-dce4430bfdda None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.387102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b48625-d66a-47dd-8534-dd1e5feda588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.387718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b48625-d66a-47dd-8534-dd1e5feda588 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.387999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b48625-d66a-47dd-8534-dd1e5feda588 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.388269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b48625-d66a-47dd-8534-dd1e5feda588 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.388966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b48625-d66a-47dd-8534-dd1e5feda588 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:56.389271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b48625-d66a-47dd-8534-dd1e5feda588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.391546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b48625-d66a-47dd-8534-dd1e5feda588 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.404399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 88/176] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.405584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb25bca-4aca-43fb-9139-10d2c00dbdf2 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.405867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb25bca-4aca-43fb-9139-10d2c00dbdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.406139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb25bca-4aca-43fb-9139-10d2c00dbdf2 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.410422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 89/177] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:56.410422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb25bca-4aca-43fb-9139-10d2c00dbdf2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.438051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.459109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 89/178] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:56.460516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821c4b5-4b46-4e91-83b4-966d14f56bf8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.460806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821c4b5-4b46-4e91-83b4-966d14f56bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.461079 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821c4b5-4b46-4e91-83b4-966d14f56bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.468353 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.483169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f821c4b5-4b46-4e91-83b4-966d14f56bf8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:56.484361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f821c4b5-4b46-4e91-83b4-966d14f56bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:56.536924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb25bca-4aca-43fb-9139-10d2c00dbdf2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.563852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 90/179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:56.564569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821c4b5-4b46-4e91-83b4-966d14f56bf8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.577243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b503603-c844-4dbc-b14d-8a5354ee0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.578455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b503603-c844-4dbc-b14d-8a5354ee0b70 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.578960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b503603-c844-4dbc-b14d-8a5354ee0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.579832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b503603-c844-4dbc-b14d-8a5354ee0b70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.592673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b503603-c844-4dbc-b14d-8a5354ee0b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6b503603-c844-4dbc-b14d-8a5354ee0b70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:56.598319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:56.599328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:13:56.600131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 90/180] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:56.601287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dee5db-d921-4fcb-aeb2-e1b5cad6d00e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.601287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dee5db-d921-4fcb-aeb2-e1b5cad6d00e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.601287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dee5db-d921-4fcb-aeb2-e1b5cad6d00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.619729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c0dee5db-d921-4fcb-aeb2-e1b5cad6d00e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:56.621224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c0dee5db-d921-4fcb-aeb2-e1b5cad6d00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:56.678119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821c4b5-4b46-4e91-83b4-966d14f56bf8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.690029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 91/181] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:56.690879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dee5db-d921-4fcb-aeb2-e1b5cad6d00e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.702156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.718690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.724202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.724823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.725096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.725364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] PATH_INFO: `/v3/users/82f38e27cd95460f986105f4aeedb6ef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.726508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] RBAC: Authorizing `identity:get_user(user_id=82f38e27cd95460f986105f4aeedb6ef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.727946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.729679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 91/182] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/users/82f38e27cd95460f986105f4aeedb6ef => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.730338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.805292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dee5db-d921-4fcb-aeb2-e1b5cad6d00e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.829983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b34eb7-e381-4d8f-9d2c-02f612f56047 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.843374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.844336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.844864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.845236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.846489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:56.848589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:56.850328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 92/183] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:56.851125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.869248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:56.869867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:56.870144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:56.870442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:56.881464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:56.886575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:56.887427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:56.888247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 92/184] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:56.888907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:56.947879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.954751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:56.971972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b9fca-0f86-4f37-931c-dd97a29fb14b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:56.998996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dda883-fa6b-4875-866c-1c81e88d1a3a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.013511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.014108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.014399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.014669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.015439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:57.015740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.017205 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.027744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 93/185] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.028465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.038594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.039231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.039508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.039771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.040886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.042324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.044183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 93/186] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.044829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.141737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8718fc0-3f86-4526-910d-54be3eff41b7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.149300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c7e22-0df4-4484-b853-2288c3289e08 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.177234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.177497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.178080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.178381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.178657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.185240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.185836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.191984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.191984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.191984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=82f38e27cd95460f986105f4aeedb6ef) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:57.191984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.191984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.192632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.195139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 94/187] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/role_assignments?user.id=82f38e27cd95460f986105f4aeedb6ef&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.195917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.279153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.280340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.337687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e07616-a24a-40a0-994c-0b1d19630994 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.387473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.388075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.388348 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.388614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.390517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:57.390820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.392178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.402091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 95/188] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.402250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 94/189] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:57.402865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.403090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.452643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.470555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.522808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd24be9-3d2e-4bbb-9fef-f396933e7ab2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.528551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd213da8-1fa7-4016-a3ca-a9013067e814 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.558196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.558858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.559135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.559405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] PATH_INFO: `/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.564064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.564907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.565378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.565837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.566807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.568334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.573014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] RBAC: Authorizing `identity:get_user(user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.575732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.576183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 95/190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.579395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.678746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.682373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9487cf-14c7-4615-af42-f607bde16e41 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.714829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.715446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.715721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.715989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] PATH_INFO: `/v3/projects/44def63210f9490082cf44969ebd9971` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.717089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] RBAC: Authorizing `identity:get_project(project_id=44def63210f9490082cf44969ebd9971)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.718586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.720199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 96/191] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/projects/44def63210f9490082cf44969ebd9971 => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.720941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.815892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43fd64-cf2f-4cfa-ac94-93eda54fc80d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.849354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.849978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.850259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.850565 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] PATH_INFO: `/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:57.851702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] RBAC: Authorizing `identity:get_user(user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:57.853145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:57.854771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 97/192] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4 => generated 283 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:57.855530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.949145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 96/193] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:13:57 2022] POST /identity/v3/users => generated 297 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:13:57.966942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:57.967078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3b24-1636-4445-b5bf-035695b3bad7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:57.967078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:57.992412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215ede3-84d7-4f63-bb53-e5bc9d3caeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:57.995065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215ede3-84d7-4f63-bb53-e5bc9d3caeae None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:57.995065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215ede3-84d7-4f63-bb53-e5bc9d3caeae None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:57.995065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215ede3-84d7-4f63-bb53-e5bc9d3caeae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.073379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215ede3-84d7-4f63-bb53-e5bc9d3caeae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.074839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215ede3-84d7-4f63-bb53-e5bc9d3caeae None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.083530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844ab72-565d-4b65-80ee-aa25da4bc174 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.095442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.113172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.124831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.125448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.126033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.126594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.127679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=44def63210f9490082cf44969ebd9971, user.id=82f38e27cd95460f986105f4aeedb6ef) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:58.128215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.130495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.136696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 97/194] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/role_assignments?user.id=82f38e27cd95460f986105f4aeedb6ef&scope.project.id=44def63210f9490082cf44969ebd9971&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:58.140078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.206430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 98/195] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:58.208764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290142c1-4651-4e85-88e3-f6b9eaeaad5c None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.209231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290142c1-4651-4e85-88e3-f6b9eaeaad5c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.209688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290142c1-4651-4e85-88e3-f6b9eaeaad5c None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.212020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 99/196] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:13:58.213174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290142c1-4651-4e85-88e3-f6b9eaeaad5c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.288006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808fa1-095c-4ae5-a3fa-1ec377e74ae6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.341235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.341771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.342074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.342319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.359302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290142c1-4651-4e85-88e3-f6b9eaeaad5c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.400141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.408575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a0d56-e25d-4171-95bb-cfa48e713598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.413373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0d56-e25d-4171-95bb-cfa48e713598 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.413373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0d56-e25d-4171-95bb-cfa48e713598 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.413373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a0d56-e25d-4171-95bb-cfa48e713598 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.413373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a0d56-e25d-4171-95bb-cfa48e713598 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.413373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a0d56-e25d-4171-95bb-cfa48e713598 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.415559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 100/197] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:58.416812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42d9cb-022e-46d4-90ad-b7e568208577 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.417097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42d9cb-022e-46d4-90ad-b7e568208577 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.417367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42d9cb-022e-46d4-90ad-b7e568208577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.423881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.431692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.434017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.440866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2f42d9cb-022e-46d4-90ad-b7e568208577 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:13:58.447738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f42d9cb-022e-46d4-90ad-b7e568208577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:13:58.520389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 101/198] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:13:58.521346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42d9cb-022e-46d4-90ad-b7e568208577 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.575139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 98/199] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:58.575988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.652199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42d9cb-022e-46d4-90ad-b7e568208577 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.654850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.680760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0d5a7-23c1-469e-a442-1a818f289bb0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.694370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:58.695236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.696028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.696307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.696691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.697529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:58.697886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.699356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.705219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 102/200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/role_assignments?user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:58.705987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.733379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.734335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.734773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.735194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.761330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:13:58.771572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:13:58.772570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:13:58.773451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 99/201] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:13:58.783324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.888330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74771c-e635-4ee1-a41b-b33c93be5798 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:58.930505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:58.931252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:58.931685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:58.931962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:58.933086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:58.934572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:58.936631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 103/202] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:58.946016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:58.998698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e721d-403c-498c-82a9-a856b5ef17e7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.040754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.076870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.077670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.078048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.078561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.079637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:59.079832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.081309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.083562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.091147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a30ac9-2c70-479c-8564-625bd9598acf None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.091313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 100/203] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:59.096870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.157521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.158128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.158456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.158733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.159460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] RBAC: Adding query filter params (role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:13:59.159765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.161345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.167126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 104/204] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:13:58 2022] GET /identity/v3/role_assignments?user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:13:59.280989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de8394-8ed3-4f3d-904d-1e4d142d73ae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.320173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.320818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.321095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.321367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] PATH_INFO: `/v3/users/35820b183a584ac787f45a974438a379` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.322590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] RBAC: Authorizing `identity:get_user(user_id=35820b183a584ac787f45a974438a379)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.324160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.326288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 101/205] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/users/35820b183a584ac787f45a974438a379 => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:13:59.331506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.512444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdddbb-5a25-4a9e-9d42-f281dfabaf4b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.551580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.559317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.559932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.560211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.560484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.653836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:13:59.655998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:13:59.791584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 105/206] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:13:59.797384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:13:59.895524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:13:59.951596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cd01e-2151-4bd2-8575-7f609bcc3a34 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:13:59.995937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb25a16-1b62-4322-aa0c-ae33a2091d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:13:59.996558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb25a16-1b62-4322-aa0c-ae33a2091d35 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:13:59.996831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb25a16-1b62-4322-aa0c-ae33a2091d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:13:59.997097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb25a16-1b62-4322-aa0c-ae33a2091d35 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:13:59.999098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb25a16-1b62-4322-aa0c-ae33a2091d35 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.000772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb25a16-1b62-4322-aa0c-ae33a2091d35 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.002702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 102/207] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:13:59 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:00.009083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.048972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.149042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73637943-48f2-48ee-ac20-550bcb06c6d6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.200520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c41a6b-575f-4493-9b64-ba21a404965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.201527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c41a6b-575f-4493-9b64-ba21a404965f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.202019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c41a6b-575f-4493-9b64-ba21a404965f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.202517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c41a6b-575f-4493-9b64-ba21a404965f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.203452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c41a6b-575f-4493-9b64-ba21a404965f None admin] RBAC: Adding query filter params (role.id=ffd19d9da2c74b20b0831c13c655ec94, scope.project.id=cf931c3358334a969b8a22fa1edbbe55, user.id=35820b183a584ac787f45a974438a379) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:00.203976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c41a6b-575f-4493-9b64-ba21a404965f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.205695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c41a6b-575f-4493-9b64-ba21a404965f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.218872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 106/208] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/role_assignments?user.id=35820b183a584ac787f45a974438a379&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=ffd19d9da2c74b20b0831c13c655ec94 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:00.527755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a4ab-ed48-4dcc-bc7c-b002e3895853 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.528127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a4ab-ed48-4dcc-bc7c-b002e3895853 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.528227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a4ab-ed48-4dcc-bc7c-b002e3895853 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.529778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 103/209] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:00.535731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77a2e-c38e-4f0b-97c2-cc139adbf07d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.536057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77a2e-c38e-4f0b-97c2-cc139adbf07d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.536331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77a2e-c38e-4f0b-97c2-cc139adbf07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.559382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d6c77a2e-c38e-4f0b-97c2-cc139adbf07d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:00.566512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d6c77a2e-c38e-4f0b-97c2-cc139adbf07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:00.581708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013565a-4453-4602-86cd-16c5e138b512 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.582813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013565a-4453-4602-86cd-16c5e138b512 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.583332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013565a-4453-4602-86cd-16c5e138b512 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.585285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 104/210] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:00.590113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d241b06-d74f-40ef-b04c-abc6d08d8fc9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.590676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d241b06-d74f-40ef-b04c-abc6d08d8fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.591150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d241b06-d74f-40ef-b04c-abc6d08d8fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.610491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6d241b06-d74f-40ef-b04c-abc6d08d8fc9 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:00.611965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6d241b06-d74f-40ef-b04c-abc6d08d8fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:00.657908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 107/211] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:00.682500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c77a2e-c38e-4f0b-97c2-cc139adbf07d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.709891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 105/212] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:00.730231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.734123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d241b06-d74f-40ef-b04c-abc6d08d8fc9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.799786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c77a2e-c38e-4f0b-97c2-cc139adbf07d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.807599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.842272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.843597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.844021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.844702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.860513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:00.861634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d241b06-d74f-40ef-b04c-abc6d08d8fc9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:00.865325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:00.866432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:00.867528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 108/213] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:00.877390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.907044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:00.908173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:00.909010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:00.909696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:00.911777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:00.914079 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:00.921518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 106/214] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:00.922587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:00.975071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:00.994607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c69096-38c9-433e-8d80-fc6b1cc45b37 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.033287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.034176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.034629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.035587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.036684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:01.037247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.038889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.053828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e20674-c472-46e3-8724-d23c0c346800 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.054145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 109/215] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:01.055288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.085463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.092298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc815256-68f5-495c-8fc5-d8f81dea4282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.096116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815256-68f5-495c-8fc5-d8f81dea4282 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.096116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815256-68f5-495c-8fc5-d8f81dea4282 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.096116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815256-68f5-495c-8fc5-d8f81dea4282 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.096116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc815256-68f5-495c-8fc5-d8f81dea4282 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.098630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc815256-68f5-495c-8fc5-d8f81dea4282 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.165007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ecc99-1b32-4f9f-82a0-63fae2ee18d0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.181255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 107/216] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:00 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:01.186327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9b102-6eb3-4363-b7b1-cb3c5a53ff35 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.186652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9b102-6eb3-4363-b7b1-cb3c5a53ff35 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.186930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9b102-6eb3-4363-b7b1-cb3c5a53ff35 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.188433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 108/217] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:01.194299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b22e7-67c5-4641-a521-db16439365cf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.195010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b22e7-67c5-4641-a521-db16439365cf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.195364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b22e7-67c5-4641-a521-db16439365cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.212576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.213526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.213996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.214534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] PATH_INFO: `/v3/users/f636c90db4e740419277dc88d962de2b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.216467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e72b22e7-67c5-4641-a521-db16439365cf None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:01.218265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e72b22e7-67c5-4641-a521-db16439365cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:01.234569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=f636c90db4e740419277dc88d962de2b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.236058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.239764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 110/218] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/users/f636c90db4e740419277dc88d962de2b => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:01.246778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.249943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.333593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 109/219] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:01.354888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b22e7-67c5-4641-a521-db16439365cf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.377337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976082d6-f019-4ec4-8814-467e5a61b7e4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.437234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffddb59-5d16-41f7-9b60-b79dede8ee64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.437851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffddb59-5d16-41f7-9b60-b79dede8ee64 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.439095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffddb59-5d16-41f7-9b60-b79dede8ee64 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.439329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffddb59-5d16-41f7-9b60-b79dede8ee64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.519635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b22e7-67c5-4641-a521-db16439365cf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.539746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.552870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffddb59-5d16-41f7-9b60-b79dede8ee64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.555253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffddb59-5d16-41f7-9b60-b79dede8ee64 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.576981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.577155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.577155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.577155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.577155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.577650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:01.581599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:01.583121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:01.583965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 110/220] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:01.590409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.698796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 111/221] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:14:01.700063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a7ab-7123-49fa-af6c-5db00945ec74 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.700337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a7ab-7123-49fa-af6c-5db00945ec74 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.700611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a7ab-7123-49fa-af6c-5db00945ec74 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.706084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 112/222] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:01.707533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07a7ab-7123-49fa-af6c-5db00945ec74 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.768046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc7ec-a3a8-42af-88f1-6cd0fc44c600 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.810712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c94e8-d3b5-473e-8fc1-8d67194ff1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.811379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c94e8-d3b5-473e-8fc1-8d67194ff1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.811627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c94e8-d3b5-473e-8fc1-8d67194ff1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.811951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c94e8-d3b5-473e-8fc1-8d67194ff1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.812629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c94e8-d3b5-473e-8fc1-8d67194ff1e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:01.812930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c94e8-d3b5-473e-8fc1-8d67194ff1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.814324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c94e8-d3b5-473e-8fc1-8d67194ff1e5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.824309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 111/223] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:01.825872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e60a33-eb7f-4823-9d09-92d693f87f1e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.826027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e60a33-eb7f-4823-9d09-92d693f87f1e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.826306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e60a33-eb7f-4823-9d09-92d693f87f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.846975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-24e60a33-eb7f-4823-9d09-92d693f87f1e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:01.848896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-24e60a33-eb7f-4823-9d09-92d693f87f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:01.855315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07a7ab-7123-49fa-af6c-5db00945ec74 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:01.857193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.889096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:01.910964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:01.911585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:01.911866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:01.912156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:01.913516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:01.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:01.916757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 113/224] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:01.920910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:01.958335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 112/225] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:01.959701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e60a33-eb7f-4823-9d09-92d693f87f1e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.063283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261770b0-1167-45f6-81c1-5e12eb151333 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.099541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e60a33-eb7f-4823-9d09-92d693f87f1e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.105713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.106337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.106639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.106907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] PATH_INFO: `/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.108095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] RBAC: Authorizing `identity:get_user(user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.109540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.114790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 114/226] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:02.115869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.144825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.145326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.145604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.145867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.146635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=f636c90db4e740419277dc88d962de2b) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:02.147226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.148756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.155570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 113/227] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:01 2022] GET /identity/v3/role_assignments?user.id=f636c90db4e740419277dc88d962de2b&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:02.163599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.180434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.233189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.319857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3e322-4d8b-44bb-abfa-023f553ea1ac None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.386118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2d746-47d4-4206-a98a-cbf11add851f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.390836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.391474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.391749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.392017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.416152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:02.423357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:02.424254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:02.425032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:02.431222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 115/228] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:02.438610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.451632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.453414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.453687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.453953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.459975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.461705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.465433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 114/229] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:02.475284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.570879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.575665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038c7a8-f6ee-4928-b05f-536d2fa96a3d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.631215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.633677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.637277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.637820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.638805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:02.639267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.640958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.664562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 116/230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:02.670105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.697038 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.700957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a51992-ec76-4cf0-b79c-2d11ffb880ef None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.748573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.749555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.749970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.750554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.755122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ccaf72485246e9bb9663727fa2f28c, user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.756854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.778571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0d379-1bdf-4643-bb21-17386426cc37 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.813826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.814479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.814764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.815038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] PATH_INFO: `/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.816238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] RBAC: Authorizing `identity:get_user(user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.817677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.819591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 117/231] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:02.824851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:02.838390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 115/232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:02 2022] PUT /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:02.855385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:02.931800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e674cb-9f0f-4608-9684-d77338c2608f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:02.969225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5172950-bdf1-4919-9460-b884ef33c22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:02.969871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5172950-bdf1-4919-9460-b884ef33c22d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:02.970153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5172950-bdf1-4919-9460-b884ef33c22d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:02.970463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5172950-bdf1-4919-9460-b884ef33c22d None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:02.971693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5172950-bdf1-4919-9460-b884ef33c22d None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:02.973769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5172950-bdf1-4919-9460-b884ef33c22d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:02.975706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 118/233] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:02 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:02.983474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.143147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca719a-6610-4bc8-b46a-aaa4c69fe084 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:03.185487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee44aa-8723-40e7-b0de-5741f07cc6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:03.186074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee44aa-8723-40e7-b0de-5741f07cc6d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.186369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee44aa-8723-40e7-b0de-5741f07cc6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.186634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee44aa-8723-40e7-b0de-5741f07cc6d6 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.188908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee44aa-8723-40e7-b0de-5741f07cc6d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=60ccaf72485246e9bb9663727fa2f28c, user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4, role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:03.190345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee44aa-8723-40e7-b0de-5741f07cc6d6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:03.269389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 116/234] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:02 2022] PUT /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:03.709656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c9669-d64e-4de7-bf02-04bb6eaa123b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.710031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c9669-d64e-4de7-bf02-04bb6eaa123b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.710229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c9669-d64e-4de7-bf02-04bb6eaa123b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.711729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 119/235] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:03.720784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7e248-6708-494b-8a74-7c4cbc657fd5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.721321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7e248-6708-494b-8a74-7c4cbc657fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.721812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7e248-6708-494b-8a74-7c4cbc657fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.747314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a3c7e248-6708-494b-8a74-7c4cbc657fd5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:03.748850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a3c7e248-6708-494b-8a74-7c4cbc657fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:03.769275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:03.845062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 117/236] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:03.865536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f42f0e-b334-49b1-a606-1b8bb2c89fa8 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.866460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f42f0e-b334-49b1-a606-1b8bb2c89fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.867217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f42f0e-b334-49b1-a606-1b8bb2c89fa8 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.868779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 120/237] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:03.880515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7e248-6708-494b-8a74-7c4cbc657fd5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:03.892506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e15d2c-7038-46c1-8f5f-e53a4c1e9269 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:03.893091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e15d2c-7038-46c1-8f5f-e53a4c1e9269 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:03.893579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e15d2c-7038-46c1-8f5f-e53a4c1e9269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:03.919102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-77e15d2c-7038-46c1-8f5f-e53a4c1e9269 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:03.921447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-77e15d2c-7038-46c1-8f5f-e53a4c1e9269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:04.061017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 121/238] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:04.075894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7e248-6708-494b-8a74-7c4cbc657fd5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.122097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.122789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.123073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.123341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.124461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:04.126028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:04.127931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 118/239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:04.134146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e15d2c-7038-46c1-8f5f-e53a4c1e9269 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.146953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.180822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:04.185312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:04.282694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689af00-d588-489d-83e8-37dde1dddda9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.359978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e15d2c-7038-46c1-8f5f-e53a4c1e9269 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.387542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.388143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.388422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.388694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.405240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde02ad2-062a-4b72-81a7-b1e0ccbe81fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.406119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde02ad2-062a-4b72-81a7-b1e0ccbe81fa None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.407688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde02ad2-062a-4b72-81a7-b1e0ccbe81fa None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.408174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde02ad2-062a-4b72-81a7-b1e0ccbe81fa None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.409996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde02ad2-062a-4b72-81a7-b1e0ccbe81fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:04.414388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde02ad2-062a-4b72-81a7-b1e0ccbe81fa None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:04.417877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:04.429740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:04.430776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:04.431556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 119/240] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:04.439460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.509449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 122/241] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:04 2022] POST /identity/v3/users => generated 293 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:04.511049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9a8be-a952-4c6b-a102-fe110cf49b90 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.511614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9a8be-a952-4c6b-a102-fe110cf49b90 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.511708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9a8be-a952-4c6b-a102-fe110cf49b90 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.513125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 123/242] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:04.522386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e197-8c11-4600-8989-d80444ddfea2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.522386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e197-8c11-4600-8989-d80444ddfea2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.522386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6e197-8c11-4600-8989-d80444ddfea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.543002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aab6e197-8c11-4600-8989-d80444ddfea2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:04.545525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aab6e197-8c11-4600-8989-d80444ddfea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:04.583767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:04.632913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 124/243] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:04.645816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c6113-a19b-415d-b9f0-2ff1b3f1e8d6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.659615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e197-8c11-4600-8989-d80444ddfea2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.677860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:04.716977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.717597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.717877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.718145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.718850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:04.719151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:04.720586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:04.742098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 120/244] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:04.747405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.826658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6e197-8c11-4600-8989-d80444ddfea2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.876988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.877719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.877902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.878178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.893187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678cbf1-840c-447c-9834-2632b752a83a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:04.897596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:04.904857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:04.905760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:04.906583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 125/245] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:04.913967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:04.960077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:04.961156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:04.961629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:04.962115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] PATH_INFO: `/v3/users/291fbe541e8046a6878268533c71eeb0` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:04.977692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:04.984557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] RBAC: Authorizing `identity:get_user(user_id=291fbe541e8046a6878268533c71eeb0)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:04.986927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:04.989813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 121/246] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/users/291fbe541e8046a6878268533c71eeb0 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:04.995790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.036905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.088628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119820-65c4-4e38-bb3d-9cf572bf2186 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.127411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86dc62-d82a-4d0c-a46a-8676d202ea6f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.139038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.139651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.139927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.140195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.140860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:05.141159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.142570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.152251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 126/247] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:05.157615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.173174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.174149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.174663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.175152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.248192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.252483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.255783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.273403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a12f11-d9d9-439c-9f69-3f9942cb8975 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.310470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.311443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.311862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.312320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] PATH_INFO: `/v3/users/f636c90db4e740419277dc88d962de2b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.313609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] RBAC: Authorizing `identity:get_user(user_id=f636c90db4e740419277dc88d962de2b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.315573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.317439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 127/248] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/users/f636c90db4e740419277dc88d962de2b => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:05.323168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.334709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.362867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 122/249] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:05.368894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.436517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80741-e31c-4563-83f2-bc11adf363e3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.467168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.471126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084098a2-b3f6-438b-96a4-68d4421e86b8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.475768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f2d27-fce1-4282-8221-c30011587dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.476642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f2d27-fce1-4282-8221-c30011587dea None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.488863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f2d27-fce1-4282-8221-c30011587dea None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.488863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f2d27-fce1-4282-8221-c30011587dea None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.488863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f2d27-fce1-4282-8221-c30011587dea None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.488863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f2d27-fce1-4282-8221-c30011587dea None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.488863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 128/250] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:05.488863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bf683-f8a9-4cdb-b388-34d07bb49433 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.489257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bf683-f8a9-4cdb-b388-34d07bb49433 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.489257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bf683-f8a9-4cdb-b388-34d07bb49433 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.490954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 129/251] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:05.492151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bf683-f8a9-4cdb-b388-34d07bb49433 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.509014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56054e-d6eb-430b-883b-ae97821fe8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.509640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56054e-d6eb-430b-883b-ae97821fe8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.509966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56054e-d6eb-430b-883b-ae97821fe8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.510238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56054e-d6eb-430b-883b-ae97821fe8a3 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.511443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56054e-d6eb-430b-883b-ae97821fe8a3 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.512874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56054e-d6eb-430b-883b-ae97821fe8a3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.514833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 123/252] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:05.515664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dc885-71f1-4133-93c7-8786f7e34885 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.515966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dc885-71f1-4133-93c7-8786f7e34885 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.516234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dc885-71f1-4133-93c7-8786f7e34885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.540476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b35dc885-71f1-4133-93c7-8786f7e34885 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:05.541666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b35dc885-71f1-4133-93c7-8786f7e34885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:05.578882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.606899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bf683-f8a9-4cdb-b388-34d07bb49433 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.611943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 124/253] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:05.612863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dc885-71f1-4133-93c7-8786f7e34885 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.645253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.646209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.646687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.647134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/f636c90db4e740419277dc88d962de2b/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.649451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] RBAC: Authorizing `identity:create_grant(project_id=60ccaf72485246e9bb9663727fa2f28c, user_id=f636c90db4e740419277dc88d962de2b, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.651229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.715067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 130/254] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:05 2022] PUT /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/f636c90db4e740419277dc88d962de2b/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:05.716252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:05.750073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:05.768892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dc885-71f1-4133-93c7-8786f7e34885 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:05.845719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36819f8-64d6-4a12-94cf-57744b62a22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:05.846335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36819f8-64d6-4a12-94cf-57744b62a22b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.846632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36819f8-64d6-4a12-94cf-57744b62a22b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.846897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36819f8-64d6-4a12-94cf-57744b62a22b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.848475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36819f8-64d6-4a12-94cf-57744b62a22b None admin] RBAC: Adding query filter params (role.id=ffd19d9da2c74b20b0831c13c655ec94, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=291fbe541e8046a6878268533c71eeb0) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:05.849023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36819f8-64d6-4a12-94cf-57744b62a22b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:05.869160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36819f8-64d6-4a12-94cf-57744b62a22b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:05.881356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 125/255] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/role_assignments?user.id=291fbe541e8046a6878268533c71eeb0&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=ffd19d9da2c74b20b0831c13c655ec94 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:05.882513 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab095b-09ca-468f-8e82-4e713a84390a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.882801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab095b-09ca-468f-8e82-4e713a84390a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.883071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab095b-09ca-468f-8e82-4e713a84390a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.884440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 126/256] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:05.896096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a02e-c421-41d2-b186-828f7e061cfa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:05.896722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a02e-c421-41d2-b186-828f7e061cfa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:05.896722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a02e-c421-41d2-b186-828f7e061cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:05.937236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9663a02e-c421-41d2-b186-828f7e061cfa None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:05.938505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9663a02e-c421-41d2-b186-828f7e061cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:05.947519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9de9f-bb2c-4ec0-9352-fd5dfa1e53da None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.007407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.014224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.014897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.015178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.015454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.030144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 127/257] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:06.031055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:06.036910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:06.037935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:06.038772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 131/258] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:06.046249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a02e-c421-41d2-b186-828f7e061cfa None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.080272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.173901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.190680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a02e-c421-41d2-b186-828f7e061cfa None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.263131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.263671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.263953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.264225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.265197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.270748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.283123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c16dc-a23d-4048-ab5b-daacabecdd63 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.288662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 128/259] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:06.311789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.338160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.338825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.339109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.339382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.350417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:06.355749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:06.356760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:06.357535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 132/260] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:06.362945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.417687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.472167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5817c-8e00-4b2d-a6eb-7d38795172e5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.508017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.508700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.508980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.509254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.509924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:06.510231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.511667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.525033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 133/261] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:06.526229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.568690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd695d-f30d-4ec2-aae3-40800ab752e6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.634074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256f554-4575-4901-bde3-1cf36fbc9555 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.658725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.671044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.671625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.671923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.672183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] PATH_INFO: `/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.673587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] RBAC: Authorizing `identity:get_user(user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.675092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.679841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 134/262] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:06.685191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.687133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.687797 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.688082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.688359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] PATH_INFO: `/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.691130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] RBAC: Authorizing `identity:get_user(user_id=5248b9a2f4a84ce2a8eb87f5d33cf6d4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.692802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.694574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 129/263] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/users/5248b9a2f4a84ce2a8eb87f5d33cf6d4 => generated 283 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:06.715416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:06.777517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.795479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c550f6-e2a6-46cf-bf60-804ec8871b53 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.841228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.841834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.842110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.842401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:06.897435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:06.920074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:06.924765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:06.927262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f055-c9fa-4b05-83bd-8c8a974c84c3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:06.988008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:06.988626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:06.988910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:06.989193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.036147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 135/264] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:07.043430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.078522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.080008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.107138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.172721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.193919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 130/265] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:14:07.199499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.232573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ac3f8-c206-47d1-9258-5bcf926053c5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.307780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157cb3c-d1e0-4d50-94b3-874d0c6f5e26 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.332553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:07.349333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.354446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.354446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.354446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.354446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.354446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.357603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.358238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.358602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.358878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.359412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 136/266] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:07.360031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.361462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.364155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.365279 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 131/267] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:07.371589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:07.562776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730647e0-e080-4916-8d50-c46bd4e4b730 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.572665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142eee1-9944-4501-b48e-d78ca525b931 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:07.617722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f9282-9de5-4bf3-b0bb-e9de21f4b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.618337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f9282-9de5-4bf3-b0bb-e9de21f4b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.618677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f9282-9de5-4bf3-b0bb-e9de21f4b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.618956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f9282-9de5-4bf3-b0bb-e9de21f4b0c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.619698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f9282-9de5-4bf3-b0bb-e9de21f4b0c7 None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:07.620011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f9282-9de5-4bf3-b0bb-e9de21f4b0c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.621461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f9282-9de5-4bf3-b0bb-e9de21f4b0c7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.642761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 132/268] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/role_assignments?user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:07.651279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89941eef-a332-4e56-8f31-1c0224411237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:07.651980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89941eef-a332-4e56-8f31-1c0224411237 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.652269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89941eef-a332-4e56-8f31-1c0224411237 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.652542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89941eef-a332-4e56-8f31-1c0224411237 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.653316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89941eef-a332-4e56-8f31-1c0224411237 None admin] RBAC: Adding query filter params (role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:07.653630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89941eef-a332-4e56-8f31-1c0224411237 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:07.655245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89941eef-a332-4e56-8f31-1c0224411237 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:07.662989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 137/269] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:07 2022] GET /identity/v3/role_assignments?user.id=5248b9a2f4a84ce2a8eb87f5d33cf6d4&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:07.948394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac74c-d8e6-4ee1-bac9-34cb232feb82 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.948904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac74c-d8e6-4ee1-bac9-34cb232feb82 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.948904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ac74c-d8e6-4ee1-bac9-34cb232feb82 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.950614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 133/270] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:07.956562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f3f27-ba52-4960-9b72-1f10812fd7d0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:07.956562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f3f27-ba52-4960-9b72-1f10812fd7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:07.957442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f3f27-ba52-4960-9b72-1f10812fd7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:07.980096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-075f3f27-ba52-4960-9b72-1f10812fd7d0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:07.981290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-075f3f27-ba52-4960-9b72-1f10812fd7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:07.989817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.058137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 138/271] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:08.084889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ac74c-d8e6-4ee1-bac9-34cb232feb82 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.142344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.198175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ac74c-d8e6-4ee1-bac9-34cb232feb82 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.232909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271eda2-07cb-4008-bfbf-d9d02623427e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.233245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271eda2-07cb-4008-bfbf-d9d02623427e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.233456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271eda2-07cb-4008-bfbf-d9d02623427e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.234857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 139/272] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:08.240789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c798-c6cc-4a58-b7a1-a96df3a06489 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.241079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c798-c6cc-4a58-b7a1-a96df3a06489 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.241398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c798-c6cc-4a58-b7a1-a96df3a06489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.242101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.242780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.243061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.243340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.244569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.246017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.247631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 134/273] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:08.255617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.275202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b726c798-c6cc-4a58-b7a1-a96df3a06489 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:08.276519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b726c798-c6cc-4a58-b7a1-a96df3a06489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:08.366141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 140/274] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:08.376793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27712cf-dd9f-44b6-82d3-e01b446c243f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.389883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c798-c6cc-4a58-b7a1-a96df3a06489 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.398831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.412776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.419354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.419354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.419354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.419354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.419354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.473029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.508897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 135/275] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:08 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:08.514463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c798-c6cc-4a58-b7a1-a96df3a06489 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.555801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a184192-ccdb-47aa-97f4-6e50d044ddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.556514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a184192-ccdb-47aa-97f4-6e50d044ddeb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.556789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a184192-ccdb-47aa-97f4-6e50d044ddeb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.557055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a184192-ccdb-47aa-97f4-6e50d044ddeb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.570204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a184192-ccdb-47aa-97f4-6e50d044ddeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6a184192-ccdb-47aa-97f4-6e50d044ddeb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:08.575629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:08.576645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:08.577413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 141/276] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:08.585123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.679689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faab68-db6d-4d19-9b35-a6b688c03292 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.680027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faab68-db6d-4d19-9b35-a6b688c03292 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.680209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7faab68-db6d-4d19-9b35-a6b688c03292 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.681554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 142/277] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:08.687520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18732a-9928-44bb-bf2c-af949f6c78b7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.687794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18732a-9928-44bb-bf2c-af949f6c78b7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.688064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18732a-9928-44bb-bf2c-af949f6c78b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.705681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adc76-69e8-478a-adbe-58ddf8035b65 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.708940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0e18732a-9928-44bb-bf2c-af949f6c78b7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:08.710248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0e18732a-9928-44bb-bf2c-af949f6c78b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:08.750990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.751527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.753027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.753027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.753027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:08.753027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.755921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.764747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 136/278] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:08.771408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.797831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 143/279] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:08.822929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.824341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18732a-9928-44bb-bf2c-af949f6c78b7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:08.900153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:08.905019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3509349-8fc1-4472-bf4b-b600432e2806 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.952924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18732a-9928-44bb-bf2c-af949f6c78b7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:08.954903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c2a9e-b691-4872-9240-b1dea00028c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:08.956072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c2a9e-b691-4872-9240-b1dea00028c4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.957257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c2a9e-b691-4872-9240-b1dea00028c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.957777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c2a9e-b691-4872-9240-b1dea00028c4 None admin] PATH_INFO: `/v3/users/314d646887df4467a76c87bf7c5b98d6` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.974315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c2a9e-b691-4872-9240-b1dea00028c4 None admin] RBAC: Authorizing `identity:get_user(user_id=314d646887df4467a76c87bf7c5b98d6)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:08.975837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c2a9e-b691-4872-9240-b1dea00028c4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:08.977518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 137/280] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/users/314d646887df4467a76c87bf7c5b98d6 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:08.980273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634fa2a-2709-4904-a0ff-1d033721c1f1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:08.980273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634fa2a-2709-4904-a0ff-1d033721c1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:08.980273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634fa2a-2709-4904-a0ff-1d033721c1f1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:08.982726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 138/281] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:08.983946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634fa2a-2709-4904-a0ff-1d033721c1f1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.006919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db21e7-2ed0-428c-900a-23a5bb3ed9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.007728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db21e7-2ed0-428c-900a-23a5bb3ed9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.008392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db21e7-2ed0-428c-900a-23a5bb3ed9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.008734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db21e7-2ed0-428c-900a-23a5bb3ed9a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.023776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db21e7-2ed0-428c-900a-23a5bb3ed9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-09db21e7-2ed0-428c-900a-23a5bb3ed9a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:09.029805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:09.030903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:09.031768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 144/282] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:09.032421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad80f7-3c9b-4086-b065-cfc1886f6648 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.032421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad80f7-3c9b-4086-b065-cfc1886f6648 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.032815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad80f7-3c9b-4086-b065-cfc1886f6648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.056953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0aad80f7-3c9b-4086-b065-cfc1886f6648 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:09.058444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0aad80f7-3c9b-4086-b065-cfc1886f6648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:09.100141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.108212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634fa2a-2709-4904-a0ff-1d033721c1f1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.149155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.149772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.150052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.150325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.151081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, user.id=314d646887df4467a76c87bf7c5b98d6, scope.system=all) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:09.151391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.151540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 145/283] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:09.152507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad80f7-3c9b-4086-b065-cfc1886f6648 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.152821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.170227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 139/284] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 02:14:08 2022] GET /identity/v3/role_assignments?user.id=314d646887df4467a76c87bf7c5b98d6&scope.system=all&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:09.182111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.231876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.325252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad80f7-3c9b-4086-b065-cfc1886f6648 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.365781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.367242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.367712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.368276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.369170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:09.369679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.371422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.384429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 146/285] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:09.390497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.415190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c61e3-cb98-448c-abe5-f1b025518615 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.477669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.478306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.478606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.478877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.504720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:09.509742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:09.510613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:09.511394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:09.512523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 140/286] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:09.517459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.555482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.584377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acaea8c-f639-434f-8487-daab86b39c11 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.610785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.639644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.640356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.640627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.640893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] PATH_INFO: `/v3/users/291fbe541e8046a6878268533c71eeb0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.642048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=291fbe541e8046a6878268533c71eeb0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.643523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.650140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 147/287] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/users/291fbe541e8046a6878268533c71eeb0 => generated 297 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:09.652030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.739489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c35f40-b1d2-4e36-9de7-f10ba2e301e2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.798878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6fbb9-5713-4699-9627-24b9be994fb2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:09.804720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.809499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.809985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.810479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.811367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:09.811859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.813455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.830769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 141/288] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:09.835538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.867373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:09.869241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:09.871101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:09.871579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:09.873838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:09.879183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:09.884532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 148/289] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:09.891290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:09.949546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:09.969866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baa890-7e66-47cb-8c4e-da35e13bca16 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.021925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.022598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.022840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.022966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.023139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] PATH_INFO: `/v3/users/f636c90db4e740419277dc88d962de2b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.027570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] RBAC: Authorizing `identity:get_user(user_id=f636c90db4e740419277dc88d962de2b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:10.029055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:10.032543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 142/290] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:09 2022] GET /identity/v3/users/f636c90db4e740419277dc88d962de2b => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:10.036699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.116285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c5b7e-475e-4420-bb49-cb10c487e6c4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.179869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.180516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.180790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.181058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/291fbe541e8046a6878268533c71eeb0/roles/ffd19d9da2c74b20b0831c13c655ec94` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.183869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] RBAC: Authorizing `identity:create_grant(project_id=60ccaf72485246e9bb9663727fa2f28c, user_id=291fbe541e8046a6878268533c71eeb0, role_id=ffd19d9da2c74b20b0831c13c655ec94)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:10.185288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:10.199513 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210e345-28c3-4a81-a9cd-9cedbfee4a1f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.234384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.235260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.235758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.236219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.258508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.262150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 149/291] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:09 2022] PUT /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c/users/291fbe541e8046a6878268533c71eeb0/roles/ffd19d9da2c74b20b0831c13c655ec94 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:10.307730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:10.309689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:10.435046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 143/292] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:10.442284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.571628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db958d4c-ff10-4a75-b6e1-e3598d3b573b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.584034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.611821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d258283-d8f0-4439-baf0-a74dde1042d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.615169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d258283-d8f0-4439-baf0-a74dde1042d8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.615169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d258283-d8f0-4439-baf0-a74dde1042d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.615169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d258283-d8f0-4439-baf0-a74dde1042d8 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.615169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d258283-d8f0-4439-baf0-a74dde1042d8 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:10.616117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d258283-d8f0-4439-baf0-a74dde1042d8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:10.618392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 150/293] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:10 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:10.623854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:10.694754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:10.766014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b8df0-8ee2-4e41-a895-79e7be24181d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:10.811161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beef47c-8d66-4dc8-aa4a-ed1b2bcaaa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:10.811765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beef47c-8d66-4dc8-aa4a-ed1b2bcaaa84 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:10.812039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beef47c-8d66-4dc8-aa4a-ed1b2bcaaa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:10.812308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beef47c-8d66-4dc8-aa4a-ed1b2bcaaa84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:10.813033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beef47c-8d66-4dc8-aa4a-ed1b2bcaaa84 None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=f636c90db4e740419277dc88d962de2b) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:10.813341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beef47c-8d66-4dc8-aa4a-ed1b2bcaaa84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:10.814798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beef47c-8d66-4dc8-aa4a-ed1b2bcaaa84 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:10.827186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 144/294] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:10 2022] GET /identity/v3/role_assignments?user.id=f636c90db4e740419277dc88d962de2b&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:11.048578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f28af8-1c8c-4f54-8f42-561cb5f2f063 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.048578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f28af8-1c8c-4f54-8f42-561cb5f2f063 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.048578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f28af8-1c8c-4f54-8f42-561cb5f2f063 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.049653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 151/295] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:11.056606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9df3-f296-4435-8189-a01cfd8f5686 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.057057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9df3-f296-4435-8189-a01cfd8f5686 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.057151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d9df3-f296-4435-8189-a01cfd8f5686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.078693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b62d9df3-f296-4435-8189-a01cfd8f5686 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:11.079874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b62d9df3-f296-4435-8189-a01cfd8f5686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:11.131184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3f23-0544-42e8-9762-183e73449d6f None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.131679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3f23-0544-42e8-9762-183e73449d6f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.131753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3f23-0544-42e8-9762-183e73449d6f None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.133201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 152/296] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:11.138288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c121b-98ac-43ca-a389-8ed21033384c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.138602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c121b-98ac-43ca-a389-8ed21033384c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.138872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c121b-98ac-43ca-a389-8ed21033384c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.161203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-964c121b-98ac-43ca-a389-8ed21033384c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:11.162458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-964c121b-98ac-43ca-a389-8ed21033384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:11.173750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 145/297] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:11.198679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d9df3-f296-4435-8189-a01cfd8f5686 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.246816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 153/298] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:11.255364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.272915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c121b-98ac-43ca-a389-8ed21033384c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.323309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d9df3-f296-4435-8189-a01cfd8f5686 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.374633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.375263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.375648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.375927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.377039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.378734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.380804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 146/299] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:11.386787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.388377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c121b-98ac-43ca-a389-8ed21033384c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.399536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.425067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.425974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.426391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.426678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.437371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:11.442486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:11.443366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:11.444163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 154/300] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:11.449345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.501977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a266b27-f9b9-4237-8e6d-f17793a5b0da None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.518693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.540572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f99615-aabc-4a81-8e10-909036333843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.541173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f99615-aabc-4a81-8e10-909036333843 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.541445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f99615-aabc-4a81-8e10-909036333843 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.542147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f99615-aabc-4a81-8e10-909036333843 None admin] PATH_INFO: `/v3/groups` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.542829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f99615-aabc-4a81-8e10-909036333843 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.544688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f99615-aabc-4a81-8e10-909036333843 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.556747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6882aec-d523-4984-91e1-eeef18b0a5f9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.591752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.592276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.592574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.592826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.593466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:11.593759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.595256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.606941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 155/301] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:11.612538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.616392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 147/302] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Aug 8 02:14:11 2022] POST /identity/v3/groups => generated 229 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:11.656864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.728394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b87e56-0d96-4499-a9fc-a2aaec58b248 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.728712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b87e56-0d96-4499-a9fc-a2aaec58b248 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.728944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b87e56-0d96-4499-a9fc-a2aaec58b248 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.730599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 148/303] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:11.736034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bcc0b-3dd7-4e8b-9f08-23f3d82fe83c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.736309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bcc0b-3dd7-4e8b-9f08-23f3d82fe83c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.736593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bcc0b-3dd7-4e8b-9f08-23f3d82fe83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.745508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f8102-29ab-4c48-acda-be4a2dd7e443 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.760935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-af2bcc0b-3dd7-4e8b-9f08-23f3d82fe83c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:11.763161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-af2bcc0b-3dd7-4e8b-9f08-23f3d82fe83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:11.786166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.787017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.787345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.787612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] PATH_INFO: `/v3/users/902f2035af1b4a988406fbe82a3e56e0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.804198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=902f2035af1b4a988406fbe82a3e56e0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.805780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.807860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 156/304] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/users/902f2035af1b4a988406fbe82a3e56e0 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:11.814333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.861080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 149/305] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:11.885297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bcc0b-3dd7-4e8b-9f08-23f3d82fe83c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:11.929345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:11.943404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62109c7-ba83-4f96-8378-181525207ea0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.981666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:11.982567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:11.982863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:11.983200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:11.984010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] RBAC: Adding query filter params (role.id=ffd19d9da2c74b20b0831c13c655ec94, user.id=902f2035af1b4a988406fbe82a3e56e0, scope.system=all) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:11.984360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:11.985994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:11.992676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 157/306] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/role_assignments?user.id=902f2035af1b4a988406fbe82a3e56e0&scope.system=all&role.id=ffd19d9da2c74b20b0831c13c655ec94 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:11.995295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bcc0b-3dd7-4e8b-9f08-23f3d82fe83c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:11.997352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.043987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.044979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.045455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.045918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.058955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:12.065929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:12.067512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:12.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:12.071186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 150/307] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:12.076750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.191683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dc589-b12a-4a9a-a4e5-a59d37beb4ba None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.226530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.227133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.227406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.227671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.228330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:12.228624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.229992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.240090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 158/308] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:12.245515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.383162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5b34-51e6-4426-ae0b-6ebeb31b4415 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.418797 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1829f41-0580-466c-8781-d6431fa15d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.419425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1829f41-0580-466c-8781-d6431fa15d7e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.419706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1829f41-0580-466c-8781-d6431fa15d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.419982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1829f41-0580-466c-8781-d6431fa15d7e None admin] PATH_INFO: `/v3/users/314d646887df4467a76c87bf7c5b98d6` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.421127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1829f41-0580-466c-8781-d6431fa15d7e None admin] RBAC: Authorizing `identity:get_user(user_id=314d646887df4467a76c87bf7c5b98d6)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.422601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1829f41-0580-466c-8781-d6431fa15d7e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.424307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 151/309] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/users/314d646887df4467a76c87bf7c5b98d6 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:12.430877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.447635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.537903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec44c40-1673-4382-b19b-e0407b4b618e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.571940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.572588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.572868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.573149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] PATH_INFO: `/v3/system/users/314d646887df4467a76c87bf7c5b98d6/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.574602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=314d646887df4467a76c87bf7c5b98d6, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:12.576004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:12.624741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b960-c2bd-4d19-9d72-bf5b11678b27 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.625152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b960-c2bd-4d19-9d72-bf5b11678b27 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.625433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b960-c2bd-4d19-9d72-bf5b11678b27 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.626956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 152/310] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:12.632173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91967f55-cb5c-4a3d-855e-848e3822e92c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.632518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91967f55-cb5c-4a3d-855e-848e3822e92c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.632857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91967f55-cb5c-4a3d-855e-848e3822e92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.643404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 159/311] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Aug 8 02:14:12 2022] PUT /identity/v3/system/users/314d646887df4467a76c87bf7c5b98d6/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:12.653116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-91967f55-cb5c-4a3d-855e-848e3822e92c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:12.654486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-91967f55-cb5c-4a3d-855e-848e3822e92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:12.689374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.749101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 153/312] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:12.771206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:12.860751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:12.882767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f8153-05b3-4eb0-b5b6-0e1114b0341b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:12.930897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:12.931655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:12.932005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:12.932347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:12.943690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:12.949206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:12.950240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:12.951231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:12.957538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 160/313] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:12.959376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91967f55-cb5c-4a3d-855e-848e3822e92c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:13.063568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91967f55-cb5c-4a3d-855e-848e3822e92c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:13.099301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:13.099910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.100191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.100462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.101122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:13.101423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:13.102882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:13.111588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 154/314] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:13.117742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:13.224431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdbaf8-507b-4019-8d81-da6f74e3b425 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:13.260740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:13.261704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.262171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.262671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] PATH_INFO: `/v3/users/291fbe541e8046a6878268533c71eeb0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.264015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] RBAC: Authorizing `identity:get_user(user_id=291fbe541e8046a6878268533c71eeb0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:13.265648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:13.267741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 161/315] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:13 2022] GET /identity/v3/users/291fbe541e8046a6878268533c71eeb0 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:13.275719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:13.314835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:13.384132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84e7fa-b502-4648-b467-36d68625d6af None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:13.419578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:13.420185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.420466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.420739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.492735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:13.494197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:13.518753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:13.602687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 155/316] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:13.604062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:13.623216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:13.707420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9bddd-d22d-4876-9f76-e149d63a6114 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:13.742763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c19a8c-b1ae-479d-ae2d-453ef98285f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:13.743743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c19a8c-b1ae-479d-ae2d-453ef98285f5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.744212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c19a8c-b1ae-479d-ae2d-453ef98285f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.744688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c19a8c-b1ae-479d-ae2d-453ef98285f5 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.746009 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c19a8c-b1ae-479d-ae2d-453ef98285f5 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:13.747660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c19a8c-b1ae-479d-ae2d-453ef98285f5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:13.749781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 162/317] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:13 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:13.756482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:13.811036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88010acc-7fc2-4c55-92c2-5882faa675ac None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.811676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88010acc-7fc2-4c55-92c2-5882faa675ac None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.812160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88010acc-7fc2-4c55-92c2-5882faa675ac None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.813923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 163/318] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:13.819385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f639-15ad-42d3-836f-3cd3310f394e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.819668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f639-15ad-42d3-836f-3cd3310f394e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.819939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f639-15ad-42d3-836f-3cd3310f394e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.841349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1d41f639-15ad-42d3-836f-3cd3310f394e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:13.842959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1d41f639-15ad-42d3-836f-3cd3310f394e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:13.869160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416c1ca-b9f0-4214-aba3-fbc45570c132 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:13.899376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:13.907292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:13.908075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:13.908365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:13.909041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:13.910374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] RBAC: Adding query filter params (role.id=ffd19d9da2c74b20b0831c13c655ec94, scope.project.id=60ccaf72485246e9bb9663727fa2f28c, user.id=291fbe541e8046a6878268533c71eeb0) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:13.910745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:13.912639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:13.928612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 156/319] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:13 2022] GET /identity/v3/role_assignments?user.id=291fbe541e8046a6878268533c71eeb0&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=ffd19d9da2c74b20b0831c13c655ec94 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:13.939749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 164/320] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:13.964042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.077567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db9205-8b6e-42ae-91f5-4062f06eb041 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.113858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.114515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.114794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.115065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.116200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.117898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.119773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 157/321] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:14.122857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c86f28-6090-4958-a8c2-a174bc4c55df None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.123148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c86f28-6090-4958-a8c2-a174bc4c55df None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.123477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c86f28-6090-4958-a8c2-a174bc4c55df None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.125069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 165/322] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:14.125270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.130330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3650e-138c-43d0-bae9-f11e2129fcb9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.130638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3650e-138c-43d0-bae9-f11e2129fcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.130909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3650e-138c-43d0-bae9-f11e2129fcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.142791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.156718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a2e3650e-138c-43d0-bae9-f11e2129fcb9 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:14.158341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a2e3650e-138c-43d0-bae9-f11e2129fcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:14.187346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.247718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 166/323] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:14.248629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa378bfe-596a-4443-9574-12eca45ed2ec None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.269478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3650e-138c-43d0-bae9-f11e2129fcb9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.285636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.286301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.286768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.287045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] PATH_INFO: `/v3/groups` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.287833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.289269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.367217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3650e-138c-43d0-bae9-f11e2129fcb9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.377333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 158/324] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Aug 8 02:14:14 2022] POST /identity/v3/groups => generated 226 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:14.404823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.405497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.405773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.406036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.408385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.419171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:14.423948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:14.424840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:14.425673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:14.431673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 167/325] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:14.431872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.548231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d2975-a66e-42ed-929d-a51fc32c07a9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.563300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.583126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.583737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.584023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.584296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.584960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:14.585264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.586736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.595852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 159/326] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:14.602233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.713726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1019b1-1c59-4dca-ba02-92433f0f1f9e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.749508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5e928-d5d2-4893-a83d-e00ea0aa8951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.750160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e928-d5d2-4893-a83d-e00ea0aa8951 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.750447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e928-d5d2-4893-a83d-e00ea0aa8951 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.750721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e928-d5d2-4893-a83d-e00ea0aa8951 None admin] PATH_INFO: `/v3/users/902f2035af1b4a988406fbe82a3e56e0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.751930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5e928-d5d2-4893-a83d-e00ea0aa8951 None admin] RBAC: Authorizing `identity:get_user(user_id=902f2035af1b4a988406fbe82a3e56e0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.753954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5e928-d5d2-4893-a83d-e00ea0aa8951 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:14.755946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 168/327] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/users/902f2035af1b4a988406fbe82a3e56e0 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:14.761401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.772173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0801d-636f-42e6-8242-509a3939fdca None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.772458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0801d-636f-42e6-8242-509a3939fdca None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.772737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0801d-636f-42e6-8242-509a3939fdca None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.774084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 169/328] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:14.779397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63640b8-43fb-40ee-b276-665438049324 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.779682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63640b8-43fb-40ee-b276-665438049324 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.779958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63640b8-43fb-40ee-b276-665438049324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.804037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f63640b8-43fb-40ee-b276-665438049324 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:14.806823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f63640b8-43fb-40ee-b276-665438049324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:14.888653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:14.892261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e754c-b7e7-4726-836d-7b2b6414b999 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:14.904732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 170/329] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:14.935473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63640b8-43fb-40ee-b276-665438049324 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:14.954475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:14.955628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:14.956172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:14.956708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] PATH_INFO: `/v3/system/users/902f2035af1b4a988406fbe82a3e56e0/roles/ffd19d9da2c74b20b0831c13c655ec94` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:14.958790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=902f2035af1b4a988406fbe82a3e56e0, role_id=ffd19d9da2c74b20b0831c13c655ec94)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:14.960401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.028713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 160/330] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Aug 8 02:14:14 2022] PUT /identity/v3/system/users/902f2035af1b4a988406fbe82a3e56e0/roles/ffd19d9da2c74b20b0831c13c655ec94 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:14:15.061959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63640b8-43fb-40ee-b276-665438049324 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.102914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.103544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.103819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.104084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.117489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:15.122445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:15.123362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:15.124178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 171/331] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:15.132629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.185138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.259415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b705d50-ab55-4a9f-a05a-412762689217 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.306889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.307505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.307786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.308059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.308735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:15.309040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.310436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.321521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 161/332] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:15.327443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.397456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.447671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565ce6d-0ca6-4a34-a6f8-fc4871f66e98 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.493145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465104ef-a36f-441f-bcf9-3486aa7958a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.494173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465104ef-a36f-441f-bcf9-3486aa7958a0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.494694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465104ef-a36f-441f-bcf9-3486aa7958a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.495181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465104ef-a36f-441f-bcf9-3486aa7958a0 None admin] PATH_INFO: `/v3/users/314d646887df4467a76c87bf7c5b98d6` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.496543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465104ef-a36f-441f-bcf9-3486aa7958a0 None admin] RBAC: Authorizing `identity:get_user(user_id=314d646887df4467a76c87bf7c5b98d6)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.498214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465104ef-a36f-441f-bcf9-3486aa7958a0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.500043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 172/333] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/users/314d646887df4467a76c87bf7c5b98d6 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:15.507452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:15.648298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace9356-b0a8-462e-8a05-21b3a2f3f3e7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:15.649977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:15.692929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80f7e5-7260-4905-a459-ff305fa85f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:15.693535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f7e5-7260-4905-a459-ff305fa85f62 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:15.693850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f7e5-7260-4905-a459-ff305fa85f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:15.694125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f7e5-7260-4905-a459-ff305fa85f62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:15.694882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80f7e5-7260-4905-a459-ff305fa85f62 None admin] RBAC: Adding query filter params (role.id=34c3a19b7c7f45d1904aad07a5cec5b1, user.id=314d646887df4467a76c87bf7c5b98d6, scope.system=all) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:15.695197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80f7e5-7260-4905-a459-ff305fa85f62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:15.696640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80f7e5-7260-4905-a459-ff305fa85f62 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:15.717939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 162/334] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 02:14:15 2022] GET /identity/v3/role_assignments?user.id=314d646887df4467a76c87bf7c5b98d6&scope.system=all&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:17.109097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb9a60-fcd0-4d44-ad85-123ac9738b2a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.109596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb9a60-fcd0-4d44-ad85-123ac9738b2a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.109670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb9a60-fcd0-4d44-ad85-123ac9738b2a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.111260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 173/335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:17.117474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1b094-5797-4808-bcd1-acd745a32e37 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.117923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1b094-5797-4808-bcd1-acd745a32e37 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.118123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1b094-5797-4808-bcd1-acd745a32e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.143374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d0e1b094-5797-4808-bcd1-acd745a32e37 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:17.144818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d0e1b094-5797-4808-bcd1-acd745a32e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:17.246636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 163/336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:17.286124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb9a60-fcd0-4d44-ad85-123ac9738b2a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.414285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb9a60-fcd0-4d44-ad85-123ac9738b2a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.439784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f17580-a803-451e-b915-9e766816b2a8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.440094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f17580-a803-451e-b915-9e766816b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.440335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f17580-a803-451e-b915-9e766816b2a8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.441995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 164/337] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:17.447112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:17.455677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f534c5-832e-482e-b2c2-3bb74338987a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.455953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f534c5-832e-482e-b2c2-3bb74338987a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.456762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f534c5-832e-482e-b2c2-3bb74338987a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.467046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.467673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.467952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.468221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.493591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:17.495953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-15f534c5-832e-482e-b2c2-3bb74338987a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:17.497215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-15f534c5-832e-482e-b2c2-3bb74338987a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:17.502570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:17.504004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:17.504778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:17.505380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 174/338] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:17.511350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.626300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 165/339] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:17.628389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c770fd-164d-40e9-8b28-2395599956bf None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.628677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c770fd-164d-40e9-8b28-2395599956bf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.629016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c770fd-164d-40e9-8b28-2395599956bf None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.630490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 166/340] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:17.637035 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38696dc-4d1d-4ebf-a7dc-4d7d1953cadc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.637316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38696dc-4d1d-4ebf-a7dc-4d7d1953cadc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.637584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38696dc-4d1d-4ebf-a7dc-4d7d1953cadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.668308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d38696dc-4d1d-4ebf-a7dc-4d7d1953cadc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:17.669514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d38696dc-4d1d-4ebf-a7dc-4d7d1953cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:17.698443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:17.702446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67134-f1e0-47e1-b97e-a421e0a09559 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.746816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.747475 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.747691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.747963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.748637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:17.749000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:17.750415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:17.763200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 175/341] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:17.764116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.796321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 167/342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:17.797664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38696dc-4d1d-4ebf-a7dc-4d7d1953cadc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:17.816294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:17.896642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6370f0-0582-4152-9921-7b1c6053ef71 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.946467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:17.947846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:17.947846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:17.947846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:17.962309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38696dc-4d1d-4ebf-a7dc-4d7d1953cadc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:17.963725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:17.969173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:17.970081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:17.970898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:17.982382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 176/343] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:17.982382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.011107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.012013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.012512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.012976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.055542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.148416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.150879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.182318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd8b80-4e45-43e2-b976-70ca72decb6b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.184076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.253704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.254555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.254838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.255113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.279398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:18.284656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:18.292216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 177/344] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:18.293025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.304561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 168/345] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:18.307049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.364784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.422061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b7490-d9b3-4d12-ba04-8102df882b75 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.459314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.459924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.460204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.460473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.461300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:18.461610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.463076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.474301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 169/346] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:18.475240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.499906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee28228-b506-40d1-84e4-21885a2ca365 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.541293 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2aa2-312a-4c9b-9840-bbf67d67c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.542251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2aa2-312a-4c9b-9840-bbf67d67c2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.542752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2aa2-312a-4c9b-9840-bbf67d67c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.543499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2aa2-312a-4c9b-9840-bbf67d67c2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.544361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e2aa2-312a-4c9b-9840-bbf67d67c2b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:18.544898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e2aa2-312a-4c9b-9840-bbf67d67c2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.546526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e2aa2-312a-4c9b-9840-bbf67d67c2b1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.556620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 178/347] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:18.558149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2f8cc-441c-406f-bf50-d0d90bb26fac None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.558658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2f8cc-441c-406f-bf50-d0d90bb26fac None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.559146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2f8cc-441c-406f-bf50-d0d90bb26fac None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.560666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 179/348] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:18.561551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2f8cc-441c-406f-bf50-d0d90bb26fac None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.590866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.599329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443100cf-6fc9-471c-8a2c-8bac45b37cc0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.617558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.645127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84126d48-c689-4518-bcd9-4469dc08897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.646257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84126d48-c689-4518-bcd9-4469dc08897c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.646578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84126d48-c689-4518-bcd9-4469dc08897c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.646851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84126d48-c689-4518-bcd9-4469dc08897c None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.648023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84126d48-c689-4518-bcd9-4469dc08897c None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.649459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84126d48-c689-4518-bcd9-4469dc08897c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.651785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 170/349] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:18.652977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8bf8a-fca4-40b7-bcef-d270bf14b998 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.653726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8bf8a-fca4-40b7-bcef-d270bf14b998 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.654000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8bf8a-fca4-40b7-bcef-d270bf14b998 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.655549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 171/350] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:18.656415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4d91-fd58-4655-952f-bc84f2dc4c46 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.656692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4d91-fd58-4655-952f-bc84f2dc4c46 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.656965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4d91-fd58-4655-952f-bc84f2dc4c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.677024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-265f4d91-fd58-4655-952f-bc84f2dc4c46 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:18.678212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-265f4d91-fd58-4655-952f-bc84f2dc4c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:18.695851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2f8cc-441c-406f-bf50-d0d90bb26fac None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.736809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f927f84-0abf-43fe-b81e-8ffb4e3b75ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.738310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f927f84-0abf-43fe-b81e-8ffb4e3b75ca None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.738880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f927f84-0abf-43fe-b81e-8ffb4e3b75ca None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.739374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f927f84-0abf-43fe-b81e-8ffb4e3b75ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.765438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 172/351] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:18.766172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4d91-fd58-4655-952f-bc84f2dc4c46 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:18.836562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f927f84-0abf-43fe-b81e-8ffb4e3b75ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:18.836729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f927f84-0abf-43fe-b81e-8ffb4e3b75ca None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:18.870240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.894969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4d91-fd58-4655-952f-bc84f2dc4c46 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:18.907918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:18.951845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:18.952963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.953505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.953983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.971580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 180/352] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:18.974271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622744e-c091-4b12-b5ab-4fc553b1ba64 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:18.974665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622744e-c091-4b12-b5ab-4fc553b1ba64 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:18.975038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622744e-c091-4b12-b5ab-4fc553b1ba64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:18.996103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8622744e-c091-4b12-b5ab-4fc553b1ba64 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:18.997630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8622744e-c091-4b12-b5ab-4fc553b1ba64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:19.058140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.059747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.084866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 181/353] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:19.085743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622744e-c091-4b12-b5ab-4fc553b1ba64 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.153724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.194474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 173/354] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:19.195332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.209210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.211522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622744e-c091-4b12-b5ab-4fc553b1ba64 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.260608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.261302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.261585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.261858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.272967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.274512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.276555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 182/355] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:19.277380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.324408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f84fe5-4b33-43e9-8c24-1470ecc6456c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.369298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.369929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.370210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.370526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.382871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:19.387659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:19.388751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:19.389673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 174/356] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:19.394675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.411125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9cac-1e51-4886-a754-ea59845ad1ff None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.432478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.455910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.456507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.457073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.457576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.458869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.460883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.463226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 183/357] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:19.464037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.479739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.515465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ce4e1-1d5c-47b7-8dc0-959850b3478c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.555928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.556653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.556863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.557134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.570934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:19.575673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:19.576555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:19.577352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 175/358] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:19.578013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.585959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b941-5f69-4237-94d5-8f2ae53b7a85 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.627197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd8c7e-48c4-480f-8457-2bb1707b56c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.627823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8c7e-48c4-480f-8457-2bb1707b56c5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.628644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8c7e-48c4-480f-8457-2bb1707b56c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.628914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8c7e-48c4-480f-8457-2bb1707b56c5 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.630137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd8c7e-48c4-480f-8457-2bb1707b56c5 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.631642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd8c7e-48c4-480f-8457-2bb1707b56c5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.634491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 184/359] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:19.634713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750815c-96c6-4ad9-bec3-3681d7d4e5a1 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.634989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750815c-96c6-4ad9-bec3-3681d7d4e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.635253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750815c-96c6-4ad9-bec3-3681d7d4e5a1 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.636562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 185/360] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:19.637181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750815c-96c6-4ad9-bec3-3681d7d4e5a1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.703629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.708420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdcc42-1eaa-41e9-b8f5-053cb89df700 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.745029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:19.752037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.752645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.752959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.756257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.756257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=cf931c3358334a969b8a22fa1edbbe55) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:19.756257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.756525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.761886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750815c-96c6-4ad9-bec3-3681d7d4e5a1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.766807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 176/361] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:19.767739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.807112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.808036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.808482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.808961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.809860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:19.810432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.812177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.824381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 186/362] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:19.825617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.896019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c805fd-ed3a-4c66-91f1-3407c228b068 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.948246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:19.948882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:19.949160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:19.949435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:19.950604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:19.952038 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:19.953713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 177/363] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:19.954462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:19.967367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21608cd-8d83-4301-9eab-950b4c845292 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:19.986237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.010538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fb27b-9131-4e6d-b665-31533fd57655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.011814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fb27b-9131-4e6d-b665-31533fd57655 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.012115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fb27b-9131-4e6d-b665-31533fd57655 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.012497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fb27b-9131-4e6d-b665-31533fd57655 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.013052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fb27b-9131-4e6d-b665-31533fd57655 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:20.013353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fb27b-9131-4e6d-b665-31533fd57655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.014855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fb27b-9131-4e6d-b665-31533fd57655 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.024195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 187/364] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:20.030003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a5ec-da76-4f27-bb4c-964f90edc2e8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.030292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a5ec-da76-4f27-bb4c-964f90edc2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.030628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a5ec-da76-4f27-bb4c-964f90edc2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.032958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.053221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aa16a5ec-da76-4f27-bb4c-964f90edc2e8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:20.054435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aa16a5ec-da76-4f27-bb4c-964f90edc2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:20.089401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ae4aa-1762-4721-a51f-e4f6fc3c2647 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.139864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.140699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 188/365] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:20.140809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.140809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.141070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.142008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a5ec-da76-4f27-bb4c-964f90edc2e8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.142184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.144187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.146386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 178/366] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:19 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:20.146859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.266472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a5ec-da76-4f27-bb4c-964f90edc2e8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.276745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.279391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed37c2-4030-4580-8595-8e86b5e15753 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.307106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.307738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.308015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.308275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] PATH_INFO: `/v3/users/902f2035af1b4a988406fbe82a3e56e0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.309836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] RBAC: Authorizing `identity:get_user(user_id=902f2035af1b4a988406fbe82a3e56e0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.311363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.313174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 189/367] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/users/902f2035af1b4a988406fbe82a3e56e0 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:20.314046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.323316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.332472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.333080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.333357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.333630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.334391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=cf931c3358334a969b8a22fa1edbbe55) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:20.334697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.336267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.342061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 179/368] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:20.342972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.449659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3c59-d79d-4bd4-bae3-0407b6141841 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.473732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d62a0-a8a3-4cd2-beb6-c591ff5b2ce6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.504384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.505181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.506592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.506713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.524866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.525714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.526202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.526678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.527606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=60ccaf72485246e9bb9663727fa2f28c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:20.528099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.529750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.538242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 180/369] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:20.539080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.569941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.604110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.606568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.620482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.683041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fd34e-0813-431b-82e2-86430e91ecd1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.733986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.734680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.734961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.735237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.748334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:20.754330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:20.755292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:20.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:20.759673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 181/370] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:20.760537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.779160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 190/371] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:20.780617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:20.880350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.917523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3e305-42e5-4fac-91c8-13ab86095738 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.921910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095b2c1-c94f-4314-be86-1ee9511cdf25 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:20.954700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:20.979313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.979929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.980213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.980486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.981231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] RBAC: Adding query filter params (role.id=ffd19d9da2c74b20b0831c13c655ec94, user.id=902f2035af1b4a988406fbe82a3e56e0, scope.system=all) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:20.981567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.983225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.991171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:20.992051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:20.992335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:20.992610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:20.993298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:20.993598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:20.995022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:20.997952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 182/372] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/role_assignments?user.id=902f2035af1b4a988406fbe82a3e56e0&scope.system=all&role.id=ffd19d9da2c74b20b0831c13c655ec94 => generated 510 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:21.003409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.014753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 191/373] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:21.021000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.236581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf6c83-2680-4eb7-a336-9cdd061b9984 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.244390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b2cb1-bca9-4953-8fc9-27e1e565cfeb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.278014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.278758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.278964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.279241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.289103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:21.308050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.309389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.309884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.310379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.319125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.320862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.322828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 183/374] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:20 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:21.328448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.352707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.354175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.451146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:21.466397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 192/375] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:14:21.466397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.508237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:21.541791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600f5d2-60e2-4d00-9d56-46068bdb8120 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.575593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c086-8440-4c1a-90ca-a2f9355e2dba None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.590023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.591071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.591554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.592023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.593474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.596153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.598447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 184/376] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:21 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:21.604019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.615942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.617308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.617494 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.617734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] PATH_INFO: `/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.618931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] RBAC: Authorizing `identity:get_project(project_id=3e036fca31f44e57a9a6f1c124d31c6b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.621000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.622858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 193/377] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:21 2022] GET /identity/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:21.629130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.730150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6ebd3-ddd6-47bc-b5f6-e2f40533df55 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.765425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d610b-5bc8-4969-9434-329a7d078084 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.771169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:21.774779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:21.776031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.776637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.776915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.777187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.777916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=60ccaf72485246e9bb9663727fa2f28c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:21.778260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.779829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.785906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 185/378] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:21 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:21.805502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7855c5-09d1-40fc-8986-b21de98207a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.806135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7855c5-09d1-40fc-8986-b21de98207a8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.806447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7855c5-09d1-40fc-8986-b21de98207a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.806720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7855c5-09d1-40fc-8986-b21de98207a8 None admin] PATH_INFO: `/v3/groups/96bb7e30554b45fcbed811c34584f7b9` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.812201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7855c5-09d1-40fc-8986-b21de98207a8 None admin] RBAC: Authorizing `identity:get_group(group_id=96bb7e30554b45fcbed811c34584f7b9)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.813707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7855c5-09d1-40fc-8986-b21de98207a8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.816620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 194/379] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:21 2022] GET /identity/v3/groups/96bb7e30554b45fcbed811c34584f7b9 => generated 229 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:21.822750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:21.932472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da173df9-5e92-44c2-9107-f2fa7015ee8f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:21.970187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06874096-db50-4217-83e2-33bc0bf7735e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:21.970842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874096-db50-4217-83e2-33bc0bf7735e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:21.971173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874096-db50-4217-83e2-33bc0bf7735e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:21.971451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06874096-db50-4217-83e2-33bc0bf7735e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:21.972762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874096-db50-4217-83e2-33bc0bf7735e None admin] RBAC: Adding query filter params (group.id=96bb7e30554b45fcbed811c34584f7b9, role.id=1b9fb565648c49388166fe130739891f, scope.project.id=3e036fca31f44e57a9a6f1c124d31c6b) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:21.973074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874096-db50-4217-83e2-33bc0bf7735e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:21.974563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06874096-db50-4217-83e2-33bc0bf7735e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:21.981056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 186/380] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:21 2022] GET /identity/v3/role_assignments?group.id=96bb7e30554b45fcbed811c34584f7b9&scope.project.id=3e036fca31f44e57a9a6f1c124d31c6b&role.id=1b9fb565648c49388166fe130739891f => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:22.365884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8aef-51a4-4d57-965e-377007772cd8 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.366671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8aef-51a4-4d57-965e-377007772cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.366746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8aef-51a4-4d57-965e-377007772cd8 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.368376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 195/381] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:22.373564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebf779-9ed3-4bdd-888d-179dce41cffc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.373969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebf779-9ed3-4bdd-888d-179dce41cffc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.374139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebf779-9ed3-4bdd-888d-179dce41cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.396985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-69ebf779-9ed3-4bdd-888d-179dce41cffc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:22.398196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-69ebf779-9ed3-4bdd-888d-179dce41cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:22.447372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:22.489772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 187/382] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:22.524278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8aef-51a4-4d57-965e-377007772cd8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:22.660042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8aef-51a4-4d57-965e-377007772cd8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:22.701572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:22.702208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.702515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.702786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.718731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:22.724254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:22.725125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:22.725941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 196/383] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:22.735464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebf779-9ed3-4bdd-888d-179dce41cffc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:22.879801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebf779-9ed3-4bdd-888d-179dce41cffc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:22.931138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:22.932139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:22.932417 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:22.932686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:22.933359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:22.933657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:22.935036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:22.946737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 188/384] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:22.951928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:22.995356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.079553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c079f6e-4008-44f4-bb71-486d1896b99d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.130545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559d9b0-87b2-4897-b1da-a7dc9284f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.131176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559d9b0-87b2-4897-b1da-a7dc9284f0cd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.131459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559d9b0-87b2-4897-b1da-a7dc9284f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.131735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559d9b0-87b2-4897-b1da-a7dc9284f0cd None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.132868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559d9b0-87b2-4897-b1da-a7dc9284f0cd None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:23.134298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559d9b0-87b2-4897-b1da-a7dc9284f0cd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:23.135898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 197/385] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:22 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:23.141534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.182564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.266483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da592f-3322-4346-be19-bb8f0f84babc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.295891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdadec-afa8-4cf5-9459-4dc8e8795081 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.296227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdadec-afa8-4cf5-9459-4dc8e8795081 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.296978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bdadec-afa8-4cf5-9459-4dc8e8795081 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.298967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 198/386] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:23.305768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608f7a0-43a3-4b19-af71-7c760d91252d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.306058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608f7a0-43a3-4b19-af71-7c760d91252d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.306707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608f7a0-43a3-4b19-af71-7c760d91252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.328471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e538c-3295-47a9-8735-850777a7b6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.329121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e538c-3295-47a9-8735-850777a7b6fa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.329402 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e538c-3295-47a9-8735-850777a7b6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.329672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e538c-3295-47a9-8735-850777a7b6fa None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.331394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e538c-3295-47a9-8735-850777a7b6fa None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:23.332182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e608f7a0-43a3-4b19-af71-7c760d91252d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:23.333016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e538c-3295-47a9-8735-850777a7b6fa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:23.333851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e608f7a0-43a3-4b19-af71-7c760d91252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:23.335000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 189/387] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:23.336130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572baab-d416-4b1d-ad49-9d8efe22e514 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.336420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572baab-d416-4b1d-ad49-9d8efe22e514 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.336702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572baab-d416-4b1d-ad49-9d8efe22e514 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.337999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 190/388] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:23.343421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7ae78-4ddd-4bc7-a480-7b6a378c39b8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.343703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7ae78-4ddd-4bc7-a480-7b6a378c39b8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.343978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7ae78-4ddd-4bc7-a480-7b6a378c39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.363873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a0c7ae78-4ddd-4bc7-a480-7b6a378c39b8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:23.365074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a0c7ae78-4ddd-4bc7-a480-7b6a378c39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:23.423525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 199/389] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:23.424355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608f7a0-43a3-4b19-af71-7c760d91252d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.441379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 191/390] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:23.463653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7ae78-4ddd-4bc7-a480-7b6a378c39b8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.476145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.503406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.555762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608f7a0-43a3-4b19-af71-7c760d91252d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.593268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.593484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7ae78-4ddd-4bc7-a480-7b6a378c39b8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.594469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.595003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.595576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55/groups/69d6f4bf43b04337997dea454da24bbe/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.598443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] RBAC: Authorizing `identity:create_grant(project_id=cf931c3358334a969b8a22fa1edbbe55, group_id=69d6f4bf43b04337997dea454da24bbe, role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:23.600125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:23.630551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.631188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.631469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.631742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.642342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:23.647705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:23.648607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:23.649400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:23.653665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 192/391] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:23.654524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.655133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 200/392] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 02:14:23 2022] PUT /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55/groups/69d6f4bf43b04337997dea454da24bbe/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:23.656299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.781182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.796491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:23.798285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425b1ed-7dc7-4a73-80fe-0c3ad91fe0ad None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.828684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19c2ef-f877-4324-b055-05d48ce07a9d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:23.842259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.843276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.844271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.844743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.845725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:23.846234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:23.847991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:23.859118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 201/393] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:23.871844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:23.924075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:23.925484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:23.925954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:23.926458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:23.939970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:23.945982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:23.946861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:23.948031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:23.950656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 193/394] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:23.956521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.012574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e274b-f3fe-41b6-9557-0f65ebc86925 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.053126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.053759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.054037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.054311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.055463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.056986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.058716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 202/395] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:24.066120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.086637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c4e21-af9b-4253-99fb-9552e0747029 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.115173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.148039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.148716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.148999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.149523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.149998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:24.150523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.151762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.170901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 194/396] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:24.176652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.208514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.217220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fce7a9-d82a-406b-a1ea-5e6c2699dffb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.259975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.261405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.261887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.262374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.263750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.265390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.268923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 203/397] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:24.275180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.374377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d209fe4-407c-4efe-986a-4d9e0cef52a1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.410380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.417281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb164ecf-b852-45cb-bcf0-966b2b122fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.418420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb164ecf-b852-45cb-bcf0-966b2b122fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.419041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb164ecf-b852-45cb-bcf0-966b2b122fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.419458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb164ecf-b852-45cb-bcf0-966b2b122fe1 None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.421066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb164ecf-b852-45cb-bcf0-966b2b122fe1 None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.422539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb164ecf-b852-45cb-bcf0-966b2b122fe1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.424120 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 195/398] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:24.429324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255db0-48cb-49b3-87ab-28163254ea49 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.429423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9460181-5c03-4bfa-9b87-9c96ea2bea7a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.429818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255db0-48cb-49b3-87ab-28163254ea49 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.429818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da255db0-48cb-49b3-87ab-28163254ea49 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.429818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 196/399] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:24.432476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735033e-717c-4c09-a2ce-df0df81910c9 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.432766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735033e-717c-4c09-a2ce-df0df81910c9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.433047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735033e-717c-4c09-a2ce-df0df81910c9 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.434490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 197/400] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:24.435032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735033e-717c-4c09-a2ce-df0df81910c9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.469422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707744cf-5f3f-4a1a-a647-3ab453e60338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.470083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707744cf-5f3f-4a1a-a647-3ab453e60338 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.470816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707744cf-5f3f-4a1a-a647-3ab453e60338 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.470969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707744cf-5f3f-4a1a-a647-3ab453e60338 None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55/groups/69d6f4bf43b04337997dea454da24bbe/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.474501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707744cf-5f3f-4a1a-a647-3ab453e60338 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf931c3358334a969b8a22fa1edbbe55, group_id=69d6f4bf43b04337997dea454da24bbe, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.475972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707744cf-5f3f-4a1a-a647-3ab453e60338 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.525078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.550152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735033e-717c-4c09-a2ce-df0df81910c9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.566489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 204/401] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 02:14:24 2022] PUT /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55/groups/69d6f4bf43b04337997dea454da24bbe/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:24.568075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c480357-66be-4784-bdff-5c6b7b537db5 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.568370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c480357-66be-4784-bdff-5c6b7b537db5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.571586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c480357-66be-4784-bdff-5c6b7b537db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.600956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6c480357-66be-4784-bdff-5c6b7b537db5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:24.603881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6c480357-66be-4784-bdff-5c6b7b537db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:24.666608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73537a8-40dd-48e5-a801-3681af0f0b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:24.667227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73537a8-40dd-48e5-a801-3681af0f0b5f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.667512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73537a8-40dd-48e5-a801-3681af0f0b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.667793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73537a8-40dd-48e5-a801-3681af0f0b5f None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.670422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73537a8-40dd-48e5-a801-3681af0f0b5f None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:24.671948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73537a8-40dd-48e5-a801-3681af0f0b5f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:24.674376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 198/402] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:24.677252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373181b-a177-4431-ac1e-12f4717f3967 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:24.679455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373181b-a177-4431-ac1e-12f4717f3967 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:24.679743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373181b-a177-4431-ac1e-12f4717f3967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:24.725813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4373181b-a177-4431-ac1e-12f4717f3967 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:24.727067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4373181b-a177-4431-ac1e-12f4717f3967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:24.743005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 205/403] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:24.743961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c480357-66be-4784-bdff-5c6b7b537db5 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.821476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 199/404] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:24.827444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373181b-a177-4431-ac1e-12f4717f3967 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:24.930698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.953967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:24.981690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c480357-66be-4784-bdff-5c6b7b537db5 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:24.989644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373181b-a177-4431-ac1e-12f4717f3967 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.029797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.031033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.031538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.032066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c/groups/69d6f4bf43b04337997dea454da24bbe/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.034906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] RBAC: Authorizing `identity:create_grant(project_id=60ccaf72485246e9bb9663727fa2f28c, group_id=69d6f4bf43b04337997dea454da24bbe, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.036137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.036664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.037081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.037524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.037929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.050516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:25.055395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:25.067290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 200/405] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:25.068093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.142392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 206/406] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 02:14:24 2022] PUT /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c/groups/69d6f4bf43b04337997dea454da24bbe/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:25.142392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.217776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ac2b7-8469-4138-a4ba-a9eebcede1fb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.286752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.287407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.287686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.287956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.308199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.317738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:25.327996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:25.328949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:25.329748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 201/407] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:25.340698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.356692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcf785-2afa-4164-a0fb-1adbe065e27e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.402142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.403158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.403822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.404251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.405046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:25.405481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.407027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.410263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.418946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 207/408] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:25.425645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.463335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419a91-dc43-4266-af7b-b9d42773051f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.502091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.507435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.507435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.507435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.507435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:25.507435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.507435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.516501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 202/409] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:25.521677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.571345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9d2e8-07f5-41d7-8d86-b940f5930be6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.586403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.629807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093c821-6de7-4061-b920-e792a3969fe3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.647077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.647728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.648011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.648289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.649541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.651016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.652900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 208/410] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:25.658148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.667229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.667903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.668188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.668463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] PATH_INFO: `/v3/groups/96bb7e30554b45fcbed811c34584f7b9` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.669603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] RBAC: Authorizing `identity:get_group(group_id=96bb7e30554b45fcbed811c34584f7b9)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.671313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.673133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 203/411] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/groups/96bb7e30554b45fcbed811c34584f7b9 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:25.679001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.793100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cefa47-c561-4993-98e3-f60e5ae3a562 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.822427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.831870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.835109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.835109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.835109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] PATH_INFO: `/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.835109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] RBAC: Authorizing `identity:get_project(project_id=3e036fca31f44e57a9a6f1c124d31c6b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.835421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:25.838140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.838823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 204/412] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:25.844544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.904027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286794a7-b82b-4c55-a174-93a8fea40e5c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:25.960984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:25.961980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:25.962486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:25.962952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:25.964277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:25.966192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:25.968425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 209/413] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:25 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:25.973917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:25.989644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d78bd9-f999-42c5-b078-9edacce2e72f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:26.037849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254352-4f7b-4141-8a50-9e158df20222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:26.038936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254352-4f7b-4141-8a50-9e158df20222 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.039431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254352-4f7b-4141-8a50-9e158df20222 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.039941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254352-4f7b-4141-8a50-9e158df20222 None admin] PATH_INFO: `/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b/groups/96bb7e30554b45fcbed811c34584f7b9/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.042068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254352-4f7b-4141-8a50-9e158df20222 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e036fca31f44e57a9a6f1c124d31c6b, group_id=96bb7e30554b45fcbed811c34584f7b9, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:26.043765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254352-4f7b-4141-8a50-9e158df20222 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:26.128796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e1328-e775-463b-afdd-fed6d00d7492 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:26.164145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:26.164806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.165079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.165348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c/groups/69d6f4bf43b04337997dea454da24bbe/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.167509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] RBAC: Authorizing `identity:create_grant(project_id=60ccaf72485246e9bb9663727fa2f28c, group_id=69d6f4bf43b04337997dea454da24bbe, role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:26.168944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:26.171286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:26.177429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 205/414] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 02:14:25 2022] PUT /identity/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b/groups/96bb7e30554b45fcbed811c34584f7b9/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:14:26.178842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061b4bb-8e95-4540-b9ab-48e42ec22ac7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.179129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061b4bb-8e95-4540-b9ab-48e42ec22ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.179397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061b4bb-8e95-4540-b9ab-48e42ec22ac7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.180632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 206/415] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:26.186459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881a408-0312-467b-b38d-537a263d9cbb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.186777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881a408-0312-467b-b38d-537a263d9cbb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.186961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881a408-0312-467b-b38d-537a263d9cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.207777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8881a408-0312-467b-b38d-537a263d9cbb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:26.209414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8881a408-0312-467b-b38d-537a263d9cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:26.251088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 210/416] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 02:14:25 2022] PUT /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c/groups/69d6f4bf43b04337997dea454da24bbe/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:14:26.285019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:26.309023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 207/417] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:26.332374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:26.461659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0923c8-8376-4acc-bd67-8aafe6cc840f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:26.499508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:26.500138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.500416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.500681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.511701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:26.516836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:26.517697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:26.519070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:26.519785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 211/418] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:26.526983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881a408-0312-467b-b38d-537a263d9cbb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:26.642430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881a408-0312-467b-b38d-537a263d9cbb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:26.692415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339cc58-0551-49aa-9bb4-6477302167ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:26.693033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339cc58-0551-49aa-9bb4-6477302167ef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.693317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339cc58-0551-49aa-9bb4-6477302167ef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.693596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339cc58-0551-49aa-9bb4-6477302167ef None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.694272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339cc58-0551-49aa-9bb4-6477302167ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:26.694616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339cc58-0551-49aa-9bb4-6477302167ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:26.696000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339cc58-0551-49aa-9bb4-6477302167ef None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:26.704805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 208/419] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:26.710373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:26.788731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:26.849506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e52d21-e71b-435a-a1ca-9b21d5fba221 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:26.864415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285c825-052c-41b4-889f-9b80f2e797e4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.864736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285c825-052c-41b4-889f-9b80f2e797e4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.864968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285c825-052c-41b4-889f-9b80f2e797e4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.866716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 209/420] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:26.892303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4847a5f-209c-46dc-be55-0f55609b9269 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.892633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4847a5f-209c-46dc-be55-0f55609b9269 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.892913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4847a5f-209c-46dc-be55-0f55609b9269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.910468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:26.911084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:26.911363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:26.911636 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:26.926667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d4847a5f-209c-46dc-be55-0f55609b9269 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:26.927881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d4847a5f-209c-46dc-be55-0f55609b9269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:27.025584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.054377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.054377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.070179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 210/421] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:27.095284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4847a5f-209c-46dc-be55-0f55609b9269 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.153054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.226878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 212/422] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:27.234375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.262501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4847a5f-209c-46dc-be55-0f55609b9269 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.300388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.301042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.301445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.301718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.312838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:27.318506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:27.319379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:27.320165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 211/423] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:27.331991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.384579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478a52b-b1a9-409c-a9b7-159df4b7a4d2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.418749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.419305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.419586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.419869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.420998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.422476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.424439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 213/424] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:27.435178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.438906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.481816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.548337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6e008-4a2b-43f3-85ed-2a0c35753799 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.564872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ef931-d7fe-4bfc-9d45-5a77cf4f5ebf None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.604035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.605067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.605442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.605810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.607064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.608583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.614948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 214/425] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:27.617574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.649618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.650233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.650543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.650811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.651488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:27.651793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.653205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.676115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 212/426] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:27.676950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:27.751970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.763969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9cf154-5a20-4c4e-b914-7bfcc99e5839 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.793072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:27.797586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bf616-5edb-4825-b7f2-23c2a3d2e988 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:27.800640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bcdc0-6f51-4c88-9f7a-84c947560940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.801302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bcdc0-6f51-4c88-9f7a-84c947560940 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.801583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bcdc0-6f51-4c88-9f7a-84c947560940 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.801853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bcdc0-6f51-4c88-9f7a-84c947560940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.802615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bcdc0-6f51-4c88-9f7a-84c947560940 None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=cf931c3358334a969b8a22fa1edbbe55) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:27.802923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bcdc0-6f51-4c88-9f7a-84c947560940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.804370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bcdc0-6f51-4c88-9f7a-84c947560940 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:27.810716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 215/427] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:27.811899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbef2f9-df1e-4b90-833b-c8cf9d471f9b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.812182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbef2f9-df1e-4b90-833b-c8cf9d471f9b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.812448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbef2f9-df1e-4b90-833b-c8cf9d471f9b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.813882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 216/428] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:27.823848 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4ad3b-d9e9-4dc6-88c0-5fbebbada8b7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.824151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4ad3b-d9e9-4dc6-88c0-5fbebbada8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.824398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4ad3b-d9e9-4dc6-88c0-5fbebbada8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.848584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:27.849606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:27.850082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:27.850654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:27.866934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5da4ad3b-d9e9-4dc6-88c0-5fbebbada8b7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:27.868531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5da4ad3b-d9e9-4dc6-88c0-5fbebbada8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:27.961526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 217/429] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:27.967881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:27.969378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.052632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4ad3b-d9e9-4dc6-88c0-5fbebbada8b7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.102500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.108434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 213/430] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:28.114418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.179505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4ad3b-d9e9-4dc6-88c0-5fbebbada8b7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.189854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.228514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.229230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.229612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.230111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.243671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:28.244398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105cc51-0a90-4f82-ac04-cfcee03bde71 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:28.249466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:28.250399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:28.251183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 218/431] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:28.257532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.287908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.288688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.289235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.289743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] PATH_INFO: `/v3/projects/cf931c3358334a969b8a22fa1edbbe55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.291221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] RBAC: Authorizing `identity:get_project(project_id=cf931c3358334a969b8a22fa1edbbe55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.292881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.295316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 214/432] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/projects/cf931c3358334a969b8a22fa1edbbe55 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:28.307866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.383326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38410c6a-0848-4789-95fd-4180fa752600 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.400252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.429476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.430128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.430595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.430871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.431873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] RBAC: Adding query filter params (name=member) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:28.432187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.434069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.441027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d51db6-496f-47af-b581-079999f86c9d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.446393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 219/433] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:28.455068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.468920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.483036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.483517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.483796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.484066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.485192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.486676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.488843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 215/434] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:28.494887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.577134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b10d0-6f0f-4fc3-b92c-879a32fbd47a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.619914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.620904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.621975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.622500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.634077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed570d6f-c06e-4684-ac25-ac142cec3f02 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.674465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54813a4-c68b-4d0e-960f-313aabd841c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:28.675103 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54813a4-c68b-4d0e-960f-313aabd841c2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.675317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54813a4-c68b-4d0e-960f-313aabd841c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.675585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54813a4-c68b-4d0e-960f-313aabd841c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.676369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54813a4-c68b-4d0e-960f-313aabd841c2 None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=cf931c3358334a969b8a22fa1edbbe55) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:28.676675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54813a4-c68b-4d0e-960f-313aabd841c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.678259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54813a4-c68b-4d0e-960f-313aabd841c2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.681711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.686514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 216/435] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=cf931c3358334a969b8a22fa1edbbe55&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:28.687849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242960-5605-4041-bb9e-36ef2f352289 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.687990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242960-5605-4041-bb9e-36ef2f352289 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.688347 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242960-5605-4041-bb9e-36ef2f352289 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.690101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 217/436] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:28.696106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4de75-0d9a-4e58-9a1b-de920202f6f1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.698158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4de75-0d9a-4e58-9a1b-de920202f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.698158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4de75-0d9a-4e58-9a1b-de920202f6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.716213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:28.716453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-43c4de75-0d9a-4e58-9a1b-de920202f6f1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:28.717620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-43c4de75-0d9a-4e58-9a1b-de920202f6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:28.717720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:28.728315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.796226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 218/437] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:28.797892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301d75b-19ca-4180-86f0-dba964da4f28 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.798238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301d75b-19ca-4180-86f0-dba964da4f28 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.798610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301d75b-19ca-4180-86f0-dba964da4f28 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.800203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 219/438] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:28.806183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fdcf-77d6-4902-9e88-8544ed1fb358 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:28.806183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fdcf-77d6-4902-9e88-8544ed1fb358 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:28.806663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fdcf-77d6-4902-9e88-8544ed1fb358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:28.826113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-18f9fdcf-77d6-4902-9e88-8544ed1fb358 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:28.827864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-18f9fdcf-77d6-4902-9e88-8544ed1fb358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:28.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 220/439] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:28.844365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.909603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 220/440] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:28.914858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9fdcf-77d6-4902-9e88-8544ed1fb358 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:28.979868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663bb3-0e4a-4576-99c4-475ddada981e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:28.989395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:28.991469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.020710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.021438 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.021665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.022014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.032490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9fdcf-77d6-4902-9e88-8544ed1fb358 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.034293 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:29.039908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:29.040859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:29.041681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:29.042326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 221/441] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:29.042919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.082495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.083013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.083294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.083566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.084775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.086292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.088043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 221/442] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:28 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:29.088858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.175349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687057c-61b4-4f21-864c-5d4d208ca78f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.212505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.213130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.213413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.213688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.214072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cab04-b83b-4418-86aa-2bf2e515a810 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.224524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:29.229672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:29.230593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 222/443] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:29.231668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.246842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.255638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.262096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.262096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.262096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.262096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:29.262096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.262096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.262644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.267229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 222/444] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:29.268591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.338639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb806eb-6983-4d3e-8430-d2e0739e5a3f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.368328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49e289-a913-43c5-9217-23b3528b03d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.371796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.372410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.372690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.372963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.374096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.375562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.377107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 223/445] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:29.378016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.404038 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.404640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.404917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.405187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.405841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:29.406143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.407547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.416151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 223/446] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:29.416932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.467538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.474079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.480820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd773717-616d-4327-8311-59a58f749cfe None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.515088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47f46-302f-4623-b369-81319b832d74 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.516673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.517280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.517559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.517832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.548612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.549236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.549606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.549959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.550712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=34c3a19b7c7f45d1904aad07a5cec5b1, scope.project.id=60ccaf72485246e9bb9663727fa2f28c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:29.551021 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.552461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.558137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 224/447] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=34c3a19b7c7f45d1904aad07a5cec5b1 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:29.558948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.600053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.601553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.674736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c63fb4-ce63-49b8-8973-19699603931c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.693719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.712967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.713649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.713930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.714756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.721222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.730012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 224/448] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:29.735608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.799604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.801444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.854127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287cc0d-6003-4935-a411-5f4781f16c89 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:29.898100 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:29.900675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:29.900675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:29.900675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] PATH_INFO: `/v3/projects/60ccaf72485246e9bb9663727fa2f28c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:29.900675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] RBAC: Authorizing `identity:get_project(project_id=60ccaf72485246e9bb9663727fa2f28c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:29.901815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:29.903838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 225/449] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/projects/60ccaf72485246e9bb9663727fa2f28c => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:29.909248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.932120 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 225/450] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:29.937052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:29.954935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:29.976338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:30.015896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf1992-5cec-4abe-8649-5d51d3cc4c08 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.036249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78c103-6949-4c53-904c-b50f42427a64 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.048961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.050102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.050565 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.050841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] PATH_INFO: `/v3/groups/69d6f4bf43b04337997dea454da24bbe` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.051960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] RBAC: Authorizing `identity:get_group(group_id=69d6f4bf43b04337997dea454da24bbe)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.053401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.055019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 226/451] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/groups/69d6f4bf43b04337997dea454da24bbe => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:30.060844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:30.071342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.071966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.072246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.072516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] PATH_INFO: `/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.073662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] RBAC: Authorizing `identity:get_project(project_id=3e036fca31f44e57a9a6f1c124d31c6b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.075149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.076766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 226/452] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:14:29 2022] GET /identity/v3/projects/3e036fca31f44e57a9a6f1c124d31c6b => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:30.082072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:30.159697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a6b4-e177-4604-8071-1061f5aa4c5c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.164403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:30.180463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424440e-9aa2-44fc-8291-b44ad3bab6eb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.188720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:30.195279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.195827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.196104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.196371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.200201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] RBAC: Adding query filter params (group.id=69d6f4bf43b04337997dea454da24bbe, role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=60ccaf72485246e9bb9663727fa2f28c) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:30.200201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.200201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.204389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 227/453] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:30 2022] GET /identity/v3/role_assignments?group.id=69d6f4bf43b04337997dea454da24bbe&scope.project.id=60ccaf72485246e9bb9663727fa2f28c&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:30.215980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc29678-3c37-4b89-bbd9-336a9a9f5df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.216633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29678-3c37-4b89-bbd9-336a9a9f5df7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.216914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29678-3c37-4b89-bbd9-336a9a9f5df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.217228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc29678-3c37-4b89-bbd9-336a9a9f5df7 None admin] PATH_INFO: `/v3/groups/96bb7e30554b45fcbed811c34584f7b9` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.218467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc29678-3c37-4b89-bbd9-336a9a9f5df7 None admin] RBAC: Authorizing `identity:get_group(group_id=96bb7e30554b45fcbed811c34584f7b9)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.219972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc29678-3c37-4b89-bbd9-336a9a9f5df7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.222215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 227/454] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 02:14:30 2022] GET /identity/v3/groups/96bb7e30554b45fcbed811c34584f7b9 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:30.228480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:30.334762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6f90-faf4-4dd0-a41b-71b057659cc8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:30.371052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51325-f8a6-4a8f-9123-b3046fda4e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:30.371654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51325-f8a6-4a8f-9123-b3046fda4e46 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:30.371935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51325-f8a6-4a8f-9123-b3046fda4e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:30.372208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51325-f8a6-4a8f-9123-b3046fda4e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:30.372936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51325-f8a6-4a8f-9123-b3046fda4e46 None admin] RBAC: Adding query filter params (group.id=96bb7e30554b45fcbed811c34584f7b9, role.id=1b9fb565648c49388166fe130739891f, scope.project.id=3e036fca31f44e57a9a6f1c124d31c6b) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:30.373237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51325-f8a6-4a8f-9123-b3046fda4e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:30.374757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51325-f8a6-4a8f-9123-b3046fda4e46 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:30.381005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 228/455] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 02:14:30 2022] GET /identity/v3/role_assignments?group.id=96bb7e30554b45fcbed811c34584f7b9&scope.project.id=3e036fca31f44e57a9a6f1c124d31c6b&role.id=1b9fb565648c49388166fe130739891f => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:32.753269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511367d-18a7-4218-8c2d-b2ccdea3f113 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.753841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511367d-18a7-4218-8c2d-b2ccdea3f113 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.754116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511367d-18a7-4218-8c2d-b2ccdea3f113 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.755920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 228/456] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:32.761184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b369c-f792-4d62-9a1d-8b08402dfe0a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:32.761586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b369c-f792-4d62-9a1d-8b08402dfe0a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:32.761729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b369c-f792-4d62-9a1d-8b08402dfe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:32.791388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b30b369c-f792-4d62-9a1d-8b08402dfe0a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:32.792609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b30b369c-f792-4d62-9a1d-8b08402dfe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:32.856067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:32.875881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 229/457] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:32.908034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511367d-18a7-4218-8c2d-b2ccdea3f113 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.065959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511367d-18a7-4218-8c2d-b2ccdea3f113 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.083496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb1d2e-e9cf-4f0a-9120-1d63c1aedc0c None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.084104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb1d2e-e9cf-4f0a-9120-1d63c1aedc0c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.084574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb1d2e-e9cf-4f0a-9120-1d63c1aedc0c None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.090455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 230/458] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:33.100067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aafbb-1ac3-4cc6-b12e-06e7887b70ab None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.100693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aafbb-1ac3-4cc6-b12e-06e7887b70ab None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.101175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aafbb-1ac3-4cc6-b12e-06e7887b70ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.118195 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.118865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.119141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.119411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.136217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7c2aafbb-1ac3-4cc6-b12e-06e7887b70ab None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:33.137419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7c2aafbb-1ac3-4cc6-b12e-06e7887b70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:33.140221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.141667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:33.167830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:33.168802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:33.170058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:33.170058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.170058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:33.170058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 229/459] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:33.175780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.186787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.272595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 231/460] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:33.300912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2aafbb-1ac3-4cc6-b12e-06e7887b70ab None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.357868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da572677-10d2-40f3-bb34-c4b713f6075d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.405719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbc85b-8523-4742-a9e6-e66139fbcd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.406850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbc85b-8523-4742-a9e6-e66139fbcd6e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.407284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbc85b-8523-4742-a9e6-e66139fbcd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.407705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbc85b-8523-4742-a9e6-e66139fbcd6e None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.408916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbc85b-8523-4742-a9e6-e66139fbcd6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:33.409493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbc85b-8523-4742-a9e6-e66139fbcd6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.411428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbc85b-8523-4742-a9e6-e66139fbcd6e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.427628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 230/461] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:33.434544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edf8f1-0143-4a53-8b89-7dae1633f119 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.434978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edf8f1-0143-4a53-8b89-7dae1633f119 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.435401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edf8f1-0143-4a53-8b89-7dae1633f119 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.437613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 231/462] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:33.442426 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edf8f1-0143-4a53-8b89-7dae1633f119 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.447804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2aafbb-1ac3-4cc6-b12e-06e7887b70ab None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.466487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.484311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4a9a9-6e3e-4702-97a5-b72edb31b5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.485074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4a9a9-6e3e-4702-97a5-b72edb31b5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.485480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4a9a9-6e3e-4702-97a5-b72edb31b5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.485863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4a9a9-6e3e-4702-97a5-b72edb31b5d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.496999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4a9a9-6e3e-4702-97a5-b72edb31b5d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.498597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4a9a9-6e3e-4702-97a5-b72edb31b5d5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1ec4a9a9-6e3e-4702-97a5-b72edb31b5d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:33.515617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:33.516715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:33.517543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:33.517543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.517543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:33.519046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 232/463] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:33.520385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0b3be-bd35-4dbe-8cd8-c44dc499db57 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.520726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0b3be-bd35-4dbe-8cd8-c44dc499db57 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.521063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0b3be-bd35-4dbe-8cd8-c44dc499db57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.537225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.540127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-25e0b3be-bd35-4dbe-8cd8-c44dc499db57 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:33.541365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-25e0b3be-bd35-4dbe-8cd8-c44dc499db57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:33.583509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edf8f1-0143-4a53-8b89-7dae1633f119 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.613668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 233/464] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:33.614273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0b3be-bd35-4dbe-8cd8-c44dc499db57 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.626247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727d482-233d-4d3c-8927-15bcccafc691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.626883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727d482-233d-4d3c-8927-15bcccafc691 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.627216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727d482-233d-4d3c-8927-15bcccafc691 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.627528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727d482-233d-4d3c-8927-15bcccafc691 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.628513 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727d482-233d-4d3c-8927-15bcccafc691 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.629955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727d482-233d-4d3c-8927-15bcccafc691 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.719323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 232/465] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:33 2022] POST /identity/v3/users => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:33.720635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3b031-ebce-4e6c-932b-c41bfdd18ca1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.721504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3b031-ebce-4e6c-932b-c41bfdd18ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.722022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3b031-ebce-4e6c-932b-c41bfdd18ca1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.726496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 233/466] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:33.726496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3b031-ebce-4e6c-932b-c41bfdd18ca1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.734079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.737032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0b3be-bd35-4dbe-8cd8-c44dc499db57 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.777839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f143602-f73e-4cf7-b714-8814c69377d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.778541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f143602-f73e-4cf7-b714-8814c69377d4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.778738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f143602-f73e-4cf7-b714-8814c69377d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.779003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f143602-f73e-4cf7-b714-8814c69377d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.779644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f143602-f73e-4cf7-b714-8814c69377d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:33.780012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f143602-f73e-4cf7-b714-8814c69377d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.781702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f143602-f73e-4cf7-b714-8814c69377d4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.803875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 234/467] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:33.805277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae3575-6d28-44de-8a21-b1d06f3edbd0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.805561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae3575-6d28-44de-8a21-b1d06f3edbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.805826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae3575-6d28-44de-8a21-b1d06f3edbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.825345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8eae3575-6d28-44de-8a21-b1d06f3edbd0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:33.826701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8eae3575-6d28-44de-8a21-b1d06f3edbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:33.854890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3b031-ebce-4e6c-932b-c41bfdd18ca1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:33.880898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:33.898173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:33.899231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:33.899707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:33.900172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:33.910236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 235/468] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:33.910867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae3575-6d28-44de-8a21-b1d06f3edbd0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:33.915269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:33.917072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:33.935469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:33.936387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:33.937237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:33.937237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:33.937237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:33.939364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 234/469] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:33.948646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:34.017173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:34.034973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae3575-6d28-44de-8a21-b1d06f3edbd0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.081128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e535b-55fd-417b-8055-4be0b3038fff None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.086653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.087263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.087540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.087810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.088451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.090563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.119009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.119780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.120126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.120470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.121178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:34.121602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.123213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.138823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 235/470] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:34.138823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:34.162309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 236/471] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:33 2022] POST /identity/v3/users => generated 248 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:34.163822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:34.193496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:34.315437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748b2a3-edbe-4582-8524-05b1e9eb2a6e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.342046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda126-7226-4fcd-bf6c-e61bdc7812fb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.362653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea29998-4f2b-45f1-b8ca-71879d604f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.363297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29998-4f2b-45f1-b8ca-71879d604f2a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.363578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29998-4f2b-45f1-b8ca-71879d604f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.363957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29998-4f2b-45f1-b8ca-71879d604f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.376933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea29998-4f2b-45f1-b8ca-71879d604f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.379555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea29998-4f2b-45f1-b8ca-71879d604f2a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.381345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108dae-d2e5-4e39-87b0-faae88a98851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.382077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108dae-d2e5-4e39-87b0-faae88a98851 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.382496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108dae-d2e5-4e39-87b0-faae88a98851 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.382778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108dae-d2e5-4e39-87b0-faae88a98851 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.383486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108dae-d2e5-4e39-87b0-faae88a98851 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.384939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108dae-d2e5-4e39-87b0-faae88a98851 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9ea29998-4f2b-45f1-b8ca-71879d604f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:34.397847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:34.398847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:34.399684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:34.399684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:34.399684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:34.399684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 236/472] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:34.400115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec17cc-4ca9-4eb5-8185-17064fb82e68 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.400440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec17cc-4ca9-4eb5-8185-17064fb82e68 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.400635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec17cc-4ca9-4eb5-8185-17064fb82e68 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.401978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 237/473] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:34.405366 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec17cc-4ca9-4eb5-8185-17064fb82e68 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:34.492294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 237/474] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:34 2022] POST /identity/v3/users => generated 248 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:34.494015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91087-4876-4152-8b30-e83eabe80050 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.494376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91087-4876-4152-8b30-e83eabe80050 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.494746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91087-4876-4152-8b30-e83eabe80050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.501933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:34.530738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:34.536435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-dcb91087-4876-4152-8b30-e83eabe80050 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:34.537918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dcb91087-4876-4152-8b30-e83eabe80050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:34.542812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec17cc-4ca9-4eb5-8185-17064fb82e68 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.603755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.604648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.606201 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.607006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.609108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:34.609108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.610457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.628163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 238/475] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:34.633492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:34.691679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 238/476] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:34.717361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91087-4876-4152-8b30-e83eabe80050 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:34.806423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e45a32-9102-407f-b770-b2c21f62f412 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.839684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91087-4876-4152-8b30-e83eabe80050 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:34.843182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.843946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.844330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.844683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.845415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.846963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.875767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:34.881276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:34.881933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:34.882228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:34.882706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:34.883369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:34.884811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:34.930979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.sql.core [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:14:34.930979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 02:14:34.930979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [parameters: {'id': 'a2b27cc0713c48e683cd4417355f039d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 02:14:34.930979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:14:34.935327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:34.940733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 239/477] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:34 2022] POST /identity/v3/users => generated 249 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Aug 08 02:14:34.952133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:14:34.953851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application yield resource Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise self.value Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application session.commit() Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:14:34.955363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:14:34.957135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:14:34.958396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:14:34.959514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [parameters: {'id': 'a2b27cc0713c48e683cd4417355f039d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:34.960562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 02:14:34.961586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:34.964975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 239/478] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 02:14:34 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:34.972206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:35.084663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6ea28-d1c8-4f94-90cc-42e2055ba8f4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:35.121436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15875c3d-d163-42c7-afff-629f9e7644ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:35.122237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15875c3d-d163-42c7-afff-629f9e7644ec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:35.122639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15875c3d-d163-42c7-afff-629f9e7644ec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:35.123023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15875c3d-d163-42c7-afff-629f9e7644ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:35.135679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15875c3d-d163-42c7-afff-629f9e7644ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-15875c3d-d163-42c7-afff-629f9e7644ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:35.141104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:35.142192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:14:35.143576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:35.146446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 240/479] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:14:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:35.151215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:35.285890 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70070b4c-6364-46b1-ac65-68ddc917dca6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:35.321661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff59808-5d42-468f-9a62-d51702a19c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:35.322267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff59808-5d42-468f-9a62-d51702a19c73 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:35.322588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff59808-5d42-468f-9a62-d51702a19c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:35.322860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff59808-5d42-468f-9a62-d51702a19c73 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:35.323517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff59808-5d42-468f-9a62-d51702a19c73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:35.323874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff59808-5d42-468f-9a62-d51702a19c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:35.325246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff59808-5d42-468f-9a62-d51702a19c73 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:35.334480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 240/480] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:36.220733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56024-f559-4326-b0b6-5a87ee53bef5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.221720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56024-f559-4326-b0b6-5a87ee53bef5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.221796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56024-f559-4326-b0b6-5a87ee53bef5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:36.223407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 241/481] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:36.228758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280551-268f-4d58-99d9-91c1a785d454 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.229141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280551-268f-4d58-99d9-91c1a785d454 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.229317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7280551-268f-4d58-99d9-91c1a785d454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:36.247909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c7280551-268f-4d58-99d9-91c1a785d454 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:36.249079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c7280551-268f-4d58-99d9-91c1a785d454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:36.305342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:36.319218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 241/482] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:36.342029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56024-f559-4326-b0b6-5a87ee53bef5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:36.404597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:36.447662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56024-f559-4326-b0b6-5a87ee53bef5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:36.482017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4d08c-f8d1-4bf9-a8ba-54ec1aa3a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:36.482583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d08c-f8d1-4bf9-a8ba-54ec1aa3a55f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.482868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d08c-f8d1-4bf9-a8ba-54ec1aa3a55f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.483129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d08c-f8d1-4bf9-a8ba-54ec1aa3a55f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:36.494341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4d08c-f8d1-4bf9-a8ba-54ec1aa3a55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e2f4d08c-f8d1-4bf9-a8ba-54ec1aa3a55f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:36.499354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:36.500245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:36.501051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 242/483] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:36.507120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7280551-268f-4d58-99d9-91c1a785d454 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:36.533995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de1ef8-80b7-46d2-92e2-2e2e4ab66971 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.534313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de1ef8-80b7-46d2-92e2-2e2e4ab66971 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.534595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de1ef8-80b7-46d2-92e2-2e2e4ab66971 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:36.535919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 243/484] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:14:36.541059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2706a-655b-4de6-99a4-6d9cff149d76 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.541337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2706a-655b-4de6-99a4-6d9cff149d76 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.541602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2706a-655b-4de6-99a4-6d9cff149d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:36.573832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0ee2706a-655b-4de6-99a4-6d9cff149d76 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:36.575061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0ee2706a-655b-4de6-99a4-6d9cff149d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:36.651310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 244/485] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:36.678811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7280551-268f-4d58-99d9-91c1a785d454 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:36.697814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2706a-655b-4de6-99a4-6d9cff149d76 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:36.756135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:36.756669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.756944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.757205 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:36.757867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:36.758161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:36.759557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:36.769433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 242/486] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:36.774960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:36.790062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:36.843028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2706a-655b-4de6-99a4-6d9cff149d76 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:36.874732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:36.904508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:36.905136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.905413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.905682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:36.930599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:36.935774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:36.936661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:36.937440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:36.945270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 245/487] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:36.951287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:36.960799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143957d-6de1-4f67-a62d-136d26d49847 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:36.996606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:36.997238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:36.997512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:36.997779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.011043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.012521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:37.038948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:37.039893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:37.040734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:37.040734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:37.040734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:37.042254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 243/488] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:37.051290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.114029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c21ca-a015-4d83-9817-2b8c6069face None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.127882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.163915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc373a60-1a16-4f19-8e25-a2d48a4178b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.164936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc373a60-1a16-4f19-8e25-a2d48a4178b3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.165403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc373a60-1a16-4f19-8e25-a2d48a4178b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.165858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc373a60-1a16-4f19-8e25-a2d48a4178b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.166881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc373a60-1a16-4f19-8e25-a2d48a4178b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:37.167852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc373a60-1a16-4f19-8e25-a2d48a4178b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.170748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc373a60-1a16-4f19-8e25-a2d48a4178b3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.182344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 246/489] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.184260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef6d46-2235-43af-ac34-b714f4ee290b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.184902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef6d46-2235-43af-ac34-b714f4ee290b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.185395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef6d46-2235-43af-ac34-b714f4ee290b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.187490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 247/490] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:37.192461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef6d46-2235-43af-ac34-b714f4ee290b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.214882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958426f-1659-4421-8e5c-b7ad5aac5e4e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.217455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.263350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792f981-f79e-4274-98e7-7ad7c90ad3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.264805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792f981-f79e-4274-98e7-7ad7c90ad3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.265530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792f981-f79e-4274-98e7-7ad7c90ad3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.265999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792f981-f79e-4274-98e7-7ad7c90ad3c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.267318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792f981-f79e-4274-98e7-7ad7c90ad3c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:37.268828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792f981-f79e-4274-98e7-7ad7c90ad3c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.271449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792f981-f79e-4274-98e7-7ad7c90ad3c2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.291141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 244/491] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.292603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6ab6-b0f5-4637-9eb5-431bd6147fce None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.294425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6ab6-b0f5-4637-9eb5-431bd6147fce None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.295022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6ab6-b0f5-4637-9eb5-431bd6147fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.315272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef6d46-2235-43af-ac34-b714f4ee290b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.317520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-13ba6ab6-b0f5-4637-9eb5-431bd6147fce None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:37.319258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-13ba6ab6-b0f5-4637-9eb5-431bd6147fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:37.358699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.359686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.360187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.360648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.375859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.379030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:37.398431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:37.399474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:37.400315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:37.400315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:37.400315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:37.400315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 248/492] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:37.400315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.402256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 245/493] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:37.404225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba6ab6-b0f5-4637-9eb5-431bd6147fce None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.433773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.527341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.532825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6d2b5-0614-40e1-9801-edc14d3b192d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.538976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba6ab6-b0f5-4637-9eb5-431bd6147fce None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.575845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.576834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.577489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.577899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.584395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.587473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.589244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.589573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.589953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.590332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.591261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:37.591771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.597759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:37.605004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:37.605898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:37.606953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:37.614256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 249/494] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:37.615314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.619998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 246/495] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.621149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.712454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.728455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676bbf0-5e8f-4d3b-b8e1-d591f297e992 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.744478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc9d2-d664-484f-b167-a99614466152 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.766025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.766887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.767228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.767519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.778416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:37.780684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8ebc8-adef-43ba-81c9-a42774e3de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.781282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8ebc8-adef-43ba-81c9-a42774e3de8d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.781562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8ebc8-adef-43ba-81c9-a42774e3de8d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.781834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8ebc8-adef-43ba-81c9-a42774e3de8d None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.782595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8ebc8-adef-43ba-81c9-a42774e3de8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:37.782903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8ebc8-adef-43ba-81c9-a42774e3de8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:37.783006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:37.783843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:37.789631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:37.790229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8ebc8-adef-43ba-81c9-a42774e3de8d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.790296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 250/496] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:37.790296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.824980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 247/497] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.826241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bdf2d-4fbb-454b-98da-80cbf582803f None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.826560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bdf2d-4fbb-454b-98da-80cbf582803f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.826835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bdf2d-4fbb-454b-98da-80cbf582803f None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.828174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 248/498] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:37.828727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bdf2d-4fbb-454b-98da-80cbf582803f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.845772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.909113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8f872-ff39-4993-a5b2-e488d6d19107 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.946380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bdf2d-4fbb-454b-98da-80cbf582803f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:37.948097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.948800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.949144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.949489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.957896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.959396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:37.970665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:37.971796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:37.972744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 251/499] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:37.973319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:37.977813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:37.983973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f3278-d5ba-4432-b8df-113bf6680f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:37.984575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f3278-d5ba-4432-b8df-113bf6680f57 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.984852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f3278-d5ba-4432-b8df-113bf6680f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.988410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f3278-d5ba-4432-b8df-113bf6680f57 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:37.988410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f3278-d5ba-4432-b8df-113bf6680f57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:37.988410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f3278-d5ba-4432-b8df-113bf6680f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:37.988410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f3278-d5ba-4432-b8df-113bf6680f57 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:37.995889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 249/500] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:37.997096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d8522-2ead-4322-ab08-59a9c270ac4a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:37.997390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d8522-2ead-4322-ab08-59a9c270ac4a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:37.997661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d8522-2ead-4322-ab08-59a9c270ac4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.016861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-965d8522-2ead-4322-ab08-59a9c270ac4a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:38.018888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-965d8522-2ead-4322-ab08-59a9c270ac4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:38.076140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa961f-2769-4ac5-ae25-6b819b2ed236 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.084732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 250/501] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:38.085957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f121d89-b4c3-4124-b3b6-fca5e0c49191 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.086246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f121d89-b4c3-4124-b3b6-fca5e0c49191 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.086551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f121d89-b4c3-4124-b3b6-fca5e0c49191 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.087853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 251/502] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:38.088546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f121d89-b4c3-4124-b3b6-fca5e0c49191 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.102760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.114660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4459cd-ea73-4087-9a03-92d70a571e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.120126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4459cd-ea73-4087-9a03-92d70a571e91 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.120126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4459cd-ea73-4087-9a03-92d70a571e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.120126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4459cd-ea73-4087-9a03-92d70a571e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.184853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4459cd-ea73-4087-9a03-92d70a571e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.186318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4459cd-ea73-4087-9a03-92d70a571e91 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.190000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.202231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f121d89-b4c3-4124-b3b6-fca5e0c49191 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.235314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.235913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.236188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.237098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.237814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:38.238117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.239591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.257209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 252/503] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:38.258006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.288613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 252/504] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:38.289768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9d1fe-f773-45fb-b577-c1ed013c9dda None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.290055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9d1fe-f773-45fb-b577-c1ed013c9dda None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.290331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9d1fe-f773-45fb-b577-c1ed013c9dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.308080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8bf9d1fe-f773-45fb-b577-c1ed013c9dda None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:38.309254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8bf9d1fe-f773-45fb-b577-c1ed013c9dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:38.317876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.360312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5134f-3f24-4442-96ab-0fc1dcde8f18 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.374998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 253/505] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:38.375927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9d1fe-f773-45fb-b577-c1ed013c9dda None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.394625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.395269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.395535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.395829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.406037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.407525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.421852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:38.422371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:38.423236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:38.424124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:38.424124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:38.424124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:38.424124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 253/506] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:38.424348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.490167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9d1fe-f773-45fb-b577-c1ed013c9dda None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.524296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.524909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.525184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.525454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.527337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6309b38-d999-4147-9152-025ec3987a86 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.535320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:38.539618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:38.539913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:38.540894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:38.541706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 254/507] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:38.542431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.562728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.563335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.563804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.564247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.632335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.633715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.640501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.647601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e107b-3e69-4c73-a884-2a4de91fc4ec None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.680572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.681176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.681484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.681750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.691927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.693350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:38.709887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:38.711001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:38.711991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:38.711991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:38.711991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:38.711991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 255/508] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:38.711991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.736113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 254/509] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:38.737450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.750994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.809590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8152ff-3139-4b74-a2ac-416d7ee46797 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.837142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fc7aa-9004-49bc-b631-fb6285d02648 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.843311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.843933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.844213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.844485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.854888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.856315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.863295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.870555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:38.870691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:38.877598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:38.878514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:38.878514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:38.878514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:38.878694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:38.878694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:38.878694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:38.878871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 256/510] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:38.878937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:38.879009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.879076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:38.879076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:38.887681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 255/511] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:38.888432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:38.964157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:38.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf623b1-f45a-405a-9484-ff73a428cdbb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:38.992822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea57f-0b7f-4ab1-bd13-9be08d389659 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.011116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.011717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.011998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.012276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.012938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:39.013242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.014635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.022968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 257/512] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:39.023934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.026986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.027750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.028032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.028306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.028944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:39.029251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.030634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.042582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 256/513] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:39.043371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.083056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.125364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b47e-ae4c-4814-8cd1-b084888649ba None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.142055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27aab73-105d-4bfd-a78d-266fa6080c56 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.158942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.159566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.159843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.160116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.170228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.170866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.171713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.175952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.176535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.176808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.177076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.177701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:39.178003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.179392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:39.185801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:39.186750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:39.187607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:39.187607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:39.187607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:39.187607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 258/514] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:39.187896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.196062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 257/515] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:39.196062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.291876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27498640-6aef-4f9d-8f2f-9d4d636058ba None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.292343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd3cc3-a64f-4906-91a1-abd5772e97f3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.305207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.325594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.326215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.326531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.326802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.327005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.327615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.327893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.328165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.336496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.337922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.338041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.339434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:39.348926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:39.349781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:39.350701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 259/516] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:39.351246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:39.353168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:39.354199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:39.355129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:39.355129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:39.355129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:39.355129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 258/517] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:39.355335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.400612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.452664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1d54f-671e-448d-99f3-40313c007f90 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.454963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749de25-88a9-4ce7-8986-291bc6cfc96f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.485032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.485630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.485904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.486213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.487907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.488501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.488810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.489047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.489678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:39.489978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.491637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.493232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.494730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.503781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 259/518] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:39.504353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:39.506485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:39.507327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:39.508250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 260/519] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:39.508803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.538809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.605949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cb846-2f28-41f7-a19c-a25f01a3a996 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.610235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae25c8c-c84d-4783-8323-5fb2ec55b8cd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.613153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.641115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.641708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.641989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.642260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.643646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:39.643942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.645384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.645943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.646569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.646851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.647241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.647746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.649201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.681325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 261/520] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:39.682074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.712583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 260/521] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:39 2022] POST /identity/v3/users => generated 247 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:14:39.713400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.782382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.793450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9494f9-9077-482e-bbcf-1abc92010a32 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.833652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.834261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.834678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.834945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.835705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:39.836006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.837379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.838609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ac4e-03a5-40c7-abb7-11f9ca033170 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:39.854815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 262/522] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:39.856090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.883007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:39.883775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:39.884174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:39.884540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:39.892248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:39.894456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:39.906961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:39.908050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:39.909605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:39.911630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 261/523] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:39.912576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:39.945470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:39.975928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ea347-c31e-451f-a31a-e5424ac0cbfa None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.014125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.014763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.015045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.015319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.016448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:40.016760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.019471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.028175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa49ca-4546-4330-bd99-24bc68488808 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.041400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 263/524] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.042234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.056414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.072830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.073438 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.073722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.073996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.149105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.151108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.154754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69764c-856f-4b24-8d7a-267260877ee5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.172440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.192929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.193558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.193836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.194110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.202230 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.203654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:40.215870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:40.216800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 264/525] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:40.218425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.256015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 262/526] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:40.256873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.300113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.332971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb77f8-167c-4b34-91e5-22493e44e3ae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.365375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778072f8-14f6-46c6-86cf-7c4e2de8deb9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.374509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.375558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.375842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.376112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.377926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:40.378651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.380351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.388171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.394422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 265/527] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.395446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.409794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.410422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.410704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.410974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.411692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=c4cfb26c57514e2cbaa73cb036118dfd) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:40.412051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.413434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.426382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 263/528] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/role_assignments?user.id=c4cfb26c57514e2cbaa73cb036118dfd&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.426382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.543877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f85937-3509-4a3c-9faf-981fa4677183 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.553951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3510a-4e26-44f3-a74b-76bdfd001a35 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.587105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.593544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.594098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.594211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.594478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.594722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.594816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.594997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.595364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.596349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:40.596648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.598098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.611950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.613579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.622492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 266/529] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.623392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:40.632350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:40.633286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:40.634123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:40.634123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:40.634123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:40.638189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 264/530] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:40.640803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.691109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.738710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f6ca3-64f9-4ab2-a1d8-4eb3df8af5e7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.778991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe993a-47a2-4a82-a7fc-6936f69c0d91 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.784593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.785522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.786559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.787049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.787997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=1149adba4d37480b8d4fee34c77d340d) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:40.789049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.791465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:40.806661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 267/531] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/role_assignments?user.id=1149adba4d37480b8d4fee34c77d340d&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:40.806661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:40.833553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.834188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.834529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.834785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.878778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:40.948412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec28f3-2c5d-43ff-8cf4-3641a1206df6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:40.985084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:40.985693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:40.985974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:40.986279 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:40.987162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:40.987464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:40.988841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:41.001445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:41.004116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:41.014760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 268/532] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:41.023591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:41.044985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:41.125986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 265/533] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:41.135365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:41.174702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81de2f-4755-44d0-8638-90fca442cd16 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:41.202752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:41.236836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:41.237469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:41.237814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:41.238082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:41.249905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:41.251364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:41.276500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:41.277394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:41.278241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 269/534] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:41.286030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:41.358937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883b2a-20a2-4f96-a5bb-4210263cfb06 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:41.416411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058ab93-ad23-49b6-9014-eb7b69d078cc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:41.429709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:41.431128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:41.431612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:41.432082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:41.442546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:41.447258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:41.449463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:41.466501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:41.468076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:41.468929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:41.468929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:41.468929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:41.471247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 266/535] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:41.476503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:41.506407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:41.507400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:41.507878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:41.508345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:41.509331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:41.509834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:41.512322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:41.538217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 270/536] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:41.544839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:41.587945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:41.669810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a7ad0-d686-45c9-953f-78243fcdf1d6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:41.688432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abdba0-207d-448c-b375-45483222ce57 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:41.714027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:41.715012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:41.715485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:41.715967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:41.716877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=38206a70a8b24cffabac622175335154) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:41.717370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:41.719145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:41.726700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 271/537] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:41 2022] GET /identity/v3/role_assignments?user.id=38206a70a8b24cffabac622175335154&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:41.787934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45c918-dbb4-4a9f-84ec-ba25e918c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:41.788568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45c918-dbb4-4a9f-84ec-ba25e918c396 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:41.788849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45c918-dbb4-4a9f-84ec-ba25e918c396 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:41.789122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45c918-dbb4-4a9f-84ec-ba25e918c396 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:41.789853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45c918-dbb4-4a9f-84ec-ba25e918c396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:41.790158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45c918-dbb4-4a9f-84ec-ba25e918c396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:41.791613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45c918-dbb4-4a9f-84ec-ba25e918c396 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:41.810041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 267/538] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:41.816404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:41.842921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d894-2231-42de-a569-83f959a0f30e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:41.843259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d894-2231-42de-a569-83f959a0f30e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:41.843474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d894-2231-42de-a569-83f959a0f30e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:41.845142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 268/539] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:41.860236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52a052-2f2e-4497-ae92-50662201e149 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:41.860555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52a052-2f2e-4497-ae92-50662201e149 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:41.860793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52a052-2f2e-4497-ae92-50662201e149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:41.898453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7e52a052-2f2e-4497-ae92-50662201e149 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:41.899743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7e52a052-2f2e-4497-ae92-50662201e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:41.950065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:41.957526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecc1d6-64ea-4ba0-8170-9b01f22dfdb8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.002472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:42.002472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.002472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.002472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.007616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:42.010426 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:42.012311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:42.025989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:42.027104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:42.028146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:42.030344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 272/540] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:42.035432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:42.055922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 269/541] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:14:42.099598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52a052-2f2e-4497-ae92-50662201e149 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:42.149479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8024-6339-4979-9e85-b8361ac80c44 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.193499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:42.194038 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.194408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.194682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.195736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:42.196052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:42.197546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:42.213848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 273/542] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:42.219665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:42.246791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:42.333580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab73b-c01a-42bd-bddd-8e1e7ad2ca3d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.349484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52a052-2f2e-4497-ae92-50662201e149 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.372180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:42.372777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.373053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.373373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.374048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=e8839af749e44fac8a7a311b61463c10) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:42.374371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:42.375863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:42.390392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 274/543] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/role_assignments?user.id=e8839af749e44fac8a7a311b61463c10&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:42.399151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:42.399795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.400074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.400347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.413071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:42.417912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:42.418910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:42.419922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 270/544] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:42.427906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:42.543000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:42.565553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2e104-5264-4e94-bfc3-da85574e082a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.603835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:42.604426 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.604699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.604963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.605626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:42.605921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:42.607290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:42.617162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 275/545] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:42.623278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:42.671075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:42.738209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471024e-f7af-41ea-b902-bfeebaae2ffd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.775395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32a706-b28c-4030-bb81-6142d0475818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:42.776395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a706-b28c-4030-bb81-6142d0475818 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.776872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a706-b28c-4030-bb81-6142d0475818 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.777353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a706-b28c-4030-bb81-6142d0475818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.789532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32a706-b28c-4030-bb81-6142d0475818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:42.791289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32a706-b28c-4030-bb81-6142d0475818 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-ca32a706-b28c-4030-bb81-6142d0475818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:42.807204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:42.808080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:42.808900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:42.808900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:42.808900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:42.810673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 271/546] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:42.819477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:42.941339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b419f-47b9-4770-b58e-b209f766017a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.942125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b419f-47b9-4770-b58e-b209f766017a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.942954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b419f-47b9-4770-b58e-b209f766017a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.944728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 272/547] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:42.950939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cde77-4956-4508-baf0-1d8a8d064a34 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:42.951602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cde77-4956-4508-baf0-1d8a8d064a34 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:42.952078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cde77-4956-4508-baf0-1d8a8d064a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:42.977911 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b9b8-5a3b-45a5-aa32-6f55e10d42e0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:42.994696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a74cde77-4956-4508-baf0-1d8a8d064a34 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:42.995901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a74cde77-4956-4508-baf0-1d8a8d064a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:43.042527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685425b-3f58-4871-8694-012701dad29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.043135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685425b-3f58-4871-8694-012701dad29b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.043418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685425b-3f58-4871-8694-012701dad29b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.043688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685425b-3f58-4871-8694-012701dad29b None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.044329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685425b-3f58-4871-8694-012701dad29b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:43.044626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685425b-3f58-4871-8694-012701dad29b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.045985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685425b-3f58-4871-8694-012701dad29b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.071323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 276/548] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:43.073046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685425b-3f58-4871-8694-012701dad29b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.082705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.178428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685425b-3f58-4871-8694-012701dad29b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.188615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 273/549] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:43.214570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.215196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.215468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.215737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.223611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.225218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.226297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cde77-4956-4508-baf0-1d8a8d064a34 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:43.239158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:43.240318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:14:43.241478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:43.245696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 277/550] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:43.246995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.274997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.321519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.376478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2965e77-1321-46bf-aa0a-6654589fa96c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.378710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cde77-4956-4508-baf0-1d8a8d064a34 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.419252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.420218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.420698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.421203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.425824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71204c03-e59b-446b-a155-f4efaf240eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.426454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204c03-e59b-446b-a155-f4efaf240eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.426736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204c03-e59b-446b-a155-f4efaf240eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.427008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204c03-e59b-446b-a155-f4efaf240eb6 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.428024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71204c03-e59b-446b-a155-f4efaf240eb6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:43.428332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71204c03-e59b-446b-a155-f4efaf240eb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.429770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71204c03-e59b-446b-a155-f4efaf240eb6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.434530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:43.439459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:43.440523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:43.441353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:43.448708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 278/552] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:43.448860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 274/552] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:43.449893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.455722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584392ca-dd21-486b-ab5e-6061d814bd72 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.458736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584392ca-dd21-486b-ab5e-6061d814bd72 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.459085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584392ca-dd21-486b-ab5e-6061d814bd72 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.464925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 279/553] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:43.464925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584392ca-dd21-486b-ab5e-6061d814bd72 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.543032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.563169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50f458-376c-43cf-a105-72d88ec6c5d1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.604133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400bbc1-9c53-44b3-b93b-33ff2b947b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.604703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584392ca-dd21-486b-ab5e-6061d814bd72 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.605544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400bbc1-9c53-44b3-b93b-33ff2b947b23 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.606010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400bbc1-9c53-44b3-b93b-33ff2b947b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.606506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400bbc1-9c53-44b3-b93b-33ff2b947b23 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.607359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400bbc1-9c53-44b3-b93b-33ff2b947b23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:43.607863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400bbc1-9c53-44b3-b93b-33ff2b947b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.609439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400bbc1-9c53-44b3-b93b-33ff2b947b23 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.623384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 275/554] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:43.625362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2286-7093-4b2b-b7a5-b80705f315c2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.625953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2286-7093-4b2b-b7a5-b80705f315c2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.626450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2286-7093-4b2b-b7a5-b80705f315c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.628412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.645679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.646289 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.646597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.646870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.648249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-66df2286-7093-4b2b-b7a5-b80705f315c2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:43.649803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-66df2286-7093-4b2b-b7a5-b80705f315c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:43.733307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 276/555] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:43.733307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2286-7093-4b2b-b7a5-b80705f315c2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.734820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.736603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.828996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.846956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 280/556] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:43.846956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.865155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:43.869950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2286-7093-4b2b-b7a5-b80705f315c2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.909485 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.910484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.910992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.911477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.922940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:43.925173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:43.943190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:43.944139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:43.945077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:43.945077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:43.945077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:43.950416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf82032-efc8-41e6-bdf6-10f52dddf0df None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:43.950977 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 277/557] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:43.952358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:43.987180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:43.987801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:43.988135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:43.988543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:43.999756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:44.005534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:44.006990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:44.007826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:44.010640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 281/558] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:44.011494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.069620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518169-9609-487a-9aa1-e61ab360a5c8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.095611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.101597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.107532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.108158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.108437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.108713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.120500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.121063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63ab8e-33f9-46a9-ab08-2f785396253b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.121882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:44.137102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:44.138266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:44.139547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:44.139547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:44.139547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:44.142808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 278/559] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:44.143698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.159161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b219f4-de13-4624-bf84-9ddf65d50ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.159764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b219f4-de13-4624-bf84-9ddf65d50ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.160043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b219f4-de13-4624-bf84-9ddf65d50ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.160317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b219f4-de13-4624-bf84-9ddf65d50ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.160956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b219f4-de13-4624-bf84-9ddf65d50ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:44.161254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b219f4-de13-4624-bf84-9ddf65d50ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.162667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b219f4-de13-4624-bf84-9ddf65d50ac2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.178958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 282/560] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:44.178958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf3288-3e9c-4932-b04b-4fe9b682b4c7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.178958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf3288-3e9c-4932-b04b-4fe9b682b4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.178958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf3288-3e9c-4932-b04b-4fe9b682b4c7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.178958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 283/561] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:44.178958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf3288-3e9c-4932-b04b-4fe9b682b4c7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.262062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3549ef1-3b97-461b-a763-6a7571876b84 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.292393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf3288-3e9c-4932-b04b-4fe9b682b4c7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.326916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.327519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.327921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.328081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.328702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:44.329007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.330958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.337467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64510fe4-4f63-4e3f-8378-afc11fa30f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.338141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64510fe4-4f63-4e3f-8378-afc11fa30f35 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.338447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64510fe4-4f63-4e3f-8378-afc11fa30f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.338717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64510fe4-4f63-4e3f-8378-afc11fa30f35 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.339368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64510fe4-4f63-4e3f-8378-afc11fa30f35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:44.339664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64510fe4-4f63-4e3f-8378-afc11fa30f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.340624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.341004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64510fe4-4f63-4e3f-8378-afc11fa30f35 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.344451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 284/562] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:44.345064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.359966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 279/563] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:44.359966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204884fb-9684-4688-9edb-9427cc909d80 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.359966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204884fb-9684-4688-9edb-9427cc909d80 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.359966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204884fb-9684-4688-9edb-9427cc909d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.376309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-204884fb-9684-4688-9edb-9427cc909d80 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:44.378215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-204884fb-9684-4688-9edb-9427cc909d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:44.411038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.454549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 280/564] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:44.454549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204884fb-9684-4688-9edb-9427cc909d80 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.471262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ade884-7c55-4124-9c5d-30e3fb5020d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.510239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.510884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.511160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.511476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.520146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.521614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:44.533876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:44.540565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 285/565] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:44.541692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.570155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204884fb-9684-4688-9edb-9427cc909d80 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.603520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.605666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.606285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.606589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.606869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.614547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.616310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:44.627611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:44.633959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 281/566] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:44.635124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.657323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509aabc-a79a-4fb6-9948-261948a4d80c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.665757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.694525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.695379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.695912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.696184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.708679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.710108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:44.725607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:44.726804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:44.727872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:44.727872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:44.727872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:44.730489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 286/567] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:44.731444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.745660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734926d-1988-4094-ae54-4e068d014ef3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.778946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.779639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.779916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.780181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.790955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:44.795762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:44.796751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 282/568] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:44.797615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.832143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4036b5-6c17-438d-b1bf-38fe300f2bac None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.844307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.866042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd4c01-8df8-4d49-8b52-9385e42c40ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.866767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd4c01-8df8-4d49-8b52-9385e42c40ae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.867227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd4c01-8df8-4d49-8b52-9385e42c40ae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.867738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd4c01-8df8-4d49-8b52-9385e42c40ae None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.868736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd4c01-8df8-4d49-8b52-9385e42c40ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:44.869215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd4c01-8df8-4d49-8b52-9385e42c40ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.871343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd4c01-8df8-4d49-8b52-9385e42c40ae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.887062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:44.888135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 287/569] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:44.889394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642dc16-46ac-4517-81f6-ffc0fff8a341 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.889681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642dc16-46ac-4517-81f6-ffc0fff8a341 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.889947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642dc16-46ac-4517-81f6-ffc0fff8a341 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.891148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 288/570] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:44.891824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642dc16-46ac-4517-81f6-ffc0fff8a341 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.904885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b6e25-d2a4-4cc6-8e2f-0a4366895e35 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:44.940248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:44.940840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:44.941114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:44.941376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:44.942055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:44.942364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:44.943811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:44.961174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 283/571] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:44.961964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:44.999599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642dc16-46ac-4517-81f6-ffc0fff8a341 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.034377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.034984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.035259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.035844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.036143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:45.036438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.037785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.049142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 289/572] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.049831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.061331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804090f-74cf-423b-8da8-1bb760656731 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.079194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.096463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5af0b7-6f3d-45ba-ab7d-383cf0bdbd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.097066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af0b7-6f3d-45ba-ab7d-383cf0bdbd03 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.097340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af0b7-6f3d-45ba-ab7d-383cf0bdbd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.097612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af0b7-6f3d-45ba-ab7d-383cf0bdbd03 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.098268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5af0b7-6f3d-45ba-ab7d-383cf0bdbd03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:45.098599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5af0b7-6f3d-45ba-ab7d-383cf0bdbd03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.099969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5af0b7-6f3d-45ba-ab7d-383cf0bdbd03 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.107691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 284/573] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.108815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6db51d-283c-48d4-b108-1a67e9483b08 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.109103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6db51d-283c-48d4-b108-1a67e9483b08 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.109379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6db51d-283c-48d4-b108-1a67e9483b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.127054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-af6db51d-283c-48d4-b108-1a67e9483b08 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:45.128254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-af6db51d-283c-48d4-b108-1a67e9483b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:45.131824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.149816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f0d39-21a8-45b6-9b90-32442caed519 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.182647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.183260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.183540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.183812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.194952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.201799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 285/574] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:45.201799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6db51d-283c-48d4-b108-1a67e9483b08 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.202009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:45.216329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:45.217610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:45.218651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:45.218651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:45.218651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:45.218651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 290/575] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:45.218651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.301679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.304406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6db51d-283c-48d4-b108-1a67e9483b08 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.319886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87087c-932a-462e-9f11-ca6f77ce931a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.338521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.339120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.339402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.339675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.340401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=481259c751234a5a936a8b0660eb8cac) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:45.340711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.342139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.354260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 286/576] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/role_assignments?user.id=481259c751234a5a936a8b0660eb8cac&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.354260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.357952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.358646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.358916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.359184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.365127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.368305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.369784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:45.383171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:45.389128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 291/577] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:45.390378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.493261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a201c2-1a1e-4231-a862-3eb7bcb2c041 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.499483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df7c69-ed90-408c-a477-c8cd31376553 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.533716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.534362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.534653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.534923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.536673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.537291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.537812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.537812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.547713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.549319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.550313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:45.555337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:45.556249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:45.557090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 292/578] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:45.557795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.562932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:45.569954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:45.571072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:45.571958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:45.571958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:45.571958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:45.571958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 287/579] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:45.572257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.633301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.671082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede6c58-a610-46c0-ba90-36eaeb3beb4c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.684692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e6f4-8c5a-49d4-a9b2-bffeb294ffda None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.711811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.712414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.712691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.713241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.713591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:45.713888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.715285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.721536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.722754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.723130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.723493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.724746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:45.725274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.726816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.731555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 293/580] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.731555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.754796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 288/581] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.754796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.803263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.844798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecf738-3a95-4cdb-9443-82f17c381165 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.854443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92c25-5cce-45d4-9831-66508010d41f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:45.869211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:45.885916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.886929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.887281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.887619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.888344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:45.888711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.890125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.890703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:45.891374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:45.891730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:45.892006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:45.892798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:45.893108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:45.894549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:45.900145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 294/582] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.901130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:45.911100 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 289/583] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:45.912063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.016249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecb2b3-e758-4737-8033-a3e1465a03e7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.022766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51bb86-a58f-4151-b02c-f94f545657a6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.041184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.053309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.054014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.054376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.054766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.057015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.057644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.057923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.058198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.062536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.064210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.069337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.070836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:46.077462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:46.078332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:46.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:46.081135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 295/584] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:46.082050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:46.085424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:46.086338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:46.087308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:46.087308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:46.087308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:46.087308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 290/585] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:46.087793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.116536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.191069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22440178-04f8-47ab-b83b-b62492c9e47b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.196939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e82ae-3e15-4a06-842e-9f7e6db06864 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.226326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.227194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.227632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.227964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] PATH_INFO: `/v3/users/glance` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.232320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.232935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.233213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.233483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.235009 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.237145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.244778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.246189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:46.250739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:46.251773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 291/586] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:46.267601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:46.268573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:46.269457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:46.270313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:46.270313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:46.270313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:46.270313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 296/587] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:46.270313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.312639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.348583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.363378 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff28b-43e2-4859-9f25-cba0477365ed None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.382926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce2978-9130-48b5-84e7-0448fc695e02 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.399110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.410394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.410394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.410394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.410394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:46.410394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.410394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.420522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 292/588] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:46.420522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.423516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.424347 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.424629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.424904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.425546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:46.426285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.427707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.444541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 297/589] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:46.444541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.532172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7812c7-f084-43dd-bd5a-836b6dee4d90 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.538613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.544384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18de74-2d68-44dc-a40c-ce8ec41393c2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.568481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.569080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.569359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.569628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.570368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:46.570677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.572155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.579256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.580503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.580790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.581067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.581699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:46.582007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.583396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.594808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 293/590] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:46.594808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.598911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.599342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 298/591] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:46.600134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.700943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff60df-bc5d-4c28-9ec1-b1d897431b5a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.706719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac2f4b-825b-4f4a-b12a-4767be41f092 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.739428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.740082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.740614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.740886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/c4cfb26c57514e2cbaa73cb036118dfd/roles/31a3f360a76e4dd9a17451573b41967e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.741966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.742613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.742891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.743156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.752017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.753686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.757270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=c4cfb26c57514e2cbaa73cb036118dfd, role_id=31a3f360a76e4dd9a17451573b41967e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.758743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:46.764940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:46.766071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 299/592] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:46.767211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.786129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.826004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 294/593] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:46 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/c4cfb26c57514e2cbaa73cb036118dfd/roles/31a3f360a76e4dd9a17451573b41967e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:46.826004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:46.885902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d3e1-9907-4af9-b6b7-bb21e3c56b30 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.895278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:46.934480 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:46.935301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:46.935640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:46.935969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:46.956929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:46.956929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:46.970766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae84de2-8619-4cd9-99e5-50c4f040ad0d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:46.976543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:46.977716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:46.978832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:46.978832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:46.978832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:46.986381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 300/594] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:46.986381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.023801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.024697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.025194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.025640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.035153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.036672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:47.050633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:47.051622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:47.052599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 295/595] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:47.053178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.074064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:47.112703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29347c8-4094-4ac0-9d6f-bff6a90db537 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.151485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.152174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.152473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.152835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.153520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:47.153821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.155375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.170420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 301/596] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:47.176626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.187952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a08db6-323d-48de-882f-dd9a8d2f19a1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.206744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:47.234418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.235029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.235584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.235893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.236713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:47.237474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.238912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.260376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 296/597] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:47.260376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.303604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aebd70-930d-495c-a1cc-101fcfbcf423 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.360627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:47.370035 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.370660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.370935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.371199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.373736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:47.373736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.374547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.393829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 302/598] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 8 02:14:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:47.394850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.404047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfe37e-9079-4d87-9873-fe181d3791e8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.440787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cba91a-7d9a-44d4-b504-bd56a82a4b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.441647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba91a-7d9a-44d4-b504-bd56a82a4b77 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.442026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba91a-7d9a-44d4-b504-bd56a82a4b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.442441 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba91a-7d9a-44d4-b504-bd56a82a4b77 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/1149adba4d37480b8d4fee34c77d340d/roles/31a3f360a76e4dd9a17451573b41967e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.458170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cba91a-7d9a-44d4-b504-bd56a82a4b77 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=1149adba4d37480b8d4fee34c77d340d, role_id=31a3f360a76e4dd9a17451573b41967e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.459850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cba91a-7d9a-44d4-b504-bd56a82a4b77 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.499237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f931d7-6113-40ec-9031-0447b07aa5e2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.533431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2150c7-bf07-41ac-9374-21f89e1d8822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.534045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2150c7-bf07-41ac-9374-21f89e1d8822 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.534324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2150c7-bf07-41ac-9374-21f89e1d8822 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.534622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2150c7-bf07-41ac-9374-21f89e1d8822 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.543404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 297/599] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:47 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/1149adba4d37480b8d4fee34c77d340d/roles/31a3f360a76e4dd9a17451573b41967e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:47.543404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a33b62-05f9-478c-ba44-eb7d5619a1c0 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.543404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a33b62-05f9-478c-ba44-eb7d5619a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.543404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a33b62-05f9-478c-ba44-eb7d5619a1c0 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.543404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 298/600] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:47.543404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a33b62-05f9-478c-ba44-eb7d5619a1c0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.551558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2150c7-bf07-41ac-9374-21f89e1d8822 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.553178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2150c7-bf07-41ac-9374-21f89e1d8822 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3b2150c7-bf07-41ac-9374-21f89e1d8822 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:47.566084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:47.567238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:47.568465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:47.570253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 303/601] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:47.571649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3b3c3-9783-4318-814c-d1a4dfc3e6bd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.572031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3b3c3-9783-4318-814c-d1a4dfc3e6bd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.572404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3b3c3-9783-4318-814c-d1a4dfc3e6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.580168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:47.592319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c1b3b3c3-9783-4318-814c-d1a4dfc3e6bd None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:47.593760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c1b3b3c3-9783-4318-814c-d1a4dfc3e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:47.649889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:47.684359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a33b62-05f9-478c-ba44-eb7d5619a1c0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.690029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 304/602] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:47.692216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3b3c3-9783-4318-814c-d1a4dfc3e6bd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.737197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.737980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.738382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.738782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.750007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.752252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:47.773832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:47.775026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:47.776084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:47.776084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:47.776084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:47.781555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 299/603] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:47.781555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.849923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3b3c3-9783-4318-814c-d1a4dfc3e6bd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.872355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:47.889248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.889985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.893509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.893509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.893509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:47.893509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:47.893509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:47.904805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cc47e-d58c-4fdd-a19d-2b064d5dbf03 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:47.911164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 305/604] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:47.911164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:47.953273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:47.954047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:47.954540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:47.954932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:47.960290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:47.967059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:47.974054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:47.975171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:47.976175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:47.982522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 300/605] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:47.982522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.060579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c718d-1c4a-4b54-836c-44f819ef2032 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.093312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17423694-f070-4774-a1be-ff6f2b49bda9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.107545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.108553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.109460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.109877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.110956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:48.111425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.113638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.134427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.134427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.134427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.134427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/38206a70a8b24cffabac622175335154/roles/31a3f360a76e4dd9a17451573b41967e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.134828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 306/606] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:48.135781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.140133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:48.153277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=38206a70a8b24cffabac622175335154, role_id=31a3f360a76e4dd9a17451573b41967e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.158300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.238779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:48.245214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f773-3365-48be-a4d2-4f696a594ac1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.262918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 301/607] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:47 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/38206a70a8b24cffabac622175335154/roles/31a3f360a76e4dd9a17451573b41967e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:48.264501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.306548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.307432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.307856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.308851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.310007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:48.310496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.312280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.324371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 307/608] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:48.332076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.495499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd4a52-a7a1-466a-b216-be8b9ebbec9f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.511235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a3a2f-c785-4920-8d14-f3493b227c5b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.535089 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.536060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.536610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.537028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.549384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.551196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.559121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.559778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.560058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.560331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:48.567432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:48.568631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:48.569708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:48.569708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:48.569708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:48.569876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.569876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.570991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 308/609] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:48.579242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:48.584428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:48.585301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:48.586177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:48.587745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 302/610] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:48.593232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.603208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:48.707860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:48.710133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722b830-4e0f-4cbf-8a61-0a16ea26cd8a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.751998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.753168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.753652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.754192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.755094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:48.755611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.757212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.772297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 309/611] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:48.777606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.815661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977944d-39d2-4c7a-a942-7a8bc7b25183 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.868066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:48.904247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b786d17-1a59-49c0-8ea3-ea9c63200940 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:48.915361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.915704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.915988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.916263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.916969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:48.917276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.918765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.943508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:48.944270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:48.944654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:48.945025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:48.946849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 303/612] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:48.952453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:48.953068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:48.955173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:48.967978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:48.969105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:14:48.970841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:48.974377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 310/613] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:48.977970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:49.127157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2aa6d7-c0b3-499a-aff2-345fdeb0f418 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:49.134974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92a1b5-db9a-4a9d-82e6-309f0875a15b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:49.135296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:49.164439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5494c3c-e4dd-4a45-969e-c0155ba44dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:49.165428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:49.165932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5494c3c-e4dd-4a45-969e-c0155ba44dc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.166212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5494c3c-e4dd-4a45-969e-c0155ba44dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.166593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5494c3c-e4dd-4a45-969e-c0155ba44dc2 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/e8839af749e44fac8a7a311b61463c10/roles/31a3f360a76e4dd9a17451573b41967e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.171352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03102a0-aee1-462d-9666-b8b8228ee7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:49.171953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03102a0-aee1-462d-9666-b8b8228ee7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.172234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03102a0-aee1-462d-9666-b8b8228ee7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.172507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03102a0-aee1-462d-9666-b8b8228ee7f5 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.173913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03102a0-aee1-462d-9666-b8b8228ee7f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:49.174230 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03102a0-aee1-462d-9666-b8b8228ee7f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:49.175937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03102a0-aee1-462d-9666-b8b8228ee7f5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:49.187938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5494c3c-e4dd-4a45-969e-c0155ba44dc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=e8839af749e44fac8a7a311b61463c10, role_id=31a3f360a76e4dd9a17451573b41967e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:49.190068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5494c3c-e4dd-4a45-969e-c0155ba44dc2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:49.200927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 311/614] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 02:14:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:49.200927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbe52b-10db-424e-809b-6a0dc227c31d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.200927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbe52b-10db-424e-809b-6a0dc227c31d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.200927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbe52b-10db-424e-809b-6a0dc227c31d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.200927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 312/615] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:49.203336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbe52b-10db-424e-809b-6a0dc227c31d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:49.258526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 304/616] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:48 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/e8839af749e44fac8a7a311b61463c10/roles/31a3f360a76e4dd9a17451573b41967e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:14:49.260665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc520d-1497-4097-b275-824a0c2f8ac0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.261215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc520d-1497-4097-b275-824a0c2f8ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.261696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc520d-1497-4097-b275-824a0c2f8ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.287729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-edfc520d-1497-4097-b275-824a0c2f8ac0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:49.289554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-edfc520d-1497-4097-b275-824a0c2f8ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:49.326451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbe52b-10db-424e-809b-6a0dc227c31d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:49.369409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 305/617] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:49.374321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:49.375383 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.375865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.376987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.388624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:49.390490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:49.394695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc520d-1497-4097-b275-824a0c2f8ac0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:49.408861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:49.409869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:49.415111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:49.415111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:49.415111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:49.415111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 313/618] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:49.424605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:49.508168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:49.511903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:49.530973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc520d-1497-4097-b275-824a0c2f8ac0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:49.593754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a254a-a7ba-4564-870a-48ca06306f0e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:49.623078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:49.623732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.623997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.624261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.634518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:49.635651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.636144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.636589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.637398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:49.637872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:49.639483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:49.658472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 314/619] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:49.659503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:49.660607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:49.664269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:49.665193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:49.666040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:49.668223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 306/620] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:49.684695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:49.777321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:49.780939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a39b0-1612-4fb4-ae45-cb440cae6e14 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:49.822767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:49.823669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.824113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.824530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.836715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:49.838555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:49.861917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:49.863097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:49.864215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:49.869737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 315/621] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:49.874582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:49.879050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abbf8c-44d7-4ade-b96e-07b26566abb6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:49.956080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:49.956702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:49.956980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:49.957252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:49.957921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:49.958223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:49.959632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:49.979466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 307/622] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:49.985086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.008501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.065853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb394d19-7bec-4f0a-8e10-a9afc8d31574 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.102332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2f7d3-4e78-457f-a88c-d455ee68ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.102968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f7d3-4e78-457f-a88c-d455ee68ec4c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.103247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f7d3-4e78-457f-a88c-d455ee68ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.103524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f7d3-4e78-457f-a88c-d455ee68ec4c None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.104217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2f7d3-4e78-457f-a88c-d455ee68ec4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:50.104520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2f7d3-4e78-457f-a88c-d455ee68ec4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.105975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2f7d3-4e78-457f-a88c-d455ee68ec4c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.119754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 316/623] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:50.121003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e819cd-c24a-4ae9-aac5-c6a338460e7f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.121291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e819cd-c24a-4ae9-aac5-c6a338460e7f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.121565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e819cd-c24a-4ae9-aac5-c6a338460e7f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.123121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 317/624] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:50.123604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2bb3-c98b-4e85-940a-00f222e4d5da None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.126248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e819cd-c24a-4ae9-aac5-c6a338460e7f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.153913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.158928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253484f6-e1db-4bf9-a072-37c98634a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.159377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253484f6-e1db-4bf9-a072-37c98634a883 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.159650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253484f6-e1db-4bf9-a072-37c98634a883 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.159914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253484f6-e1db-4bf9-a072-37c98634a883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.171212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253484f6-e1db-4bf9-a072-37c98634a883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.172661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253484f6-e1db-4bf9-a072-37c98634a883 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-253484f6-e1db-4bf9-a072-37c98634a883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:50.188130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:50.189973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:50.191011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:50.191011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:50.191011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:50.191011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 308/625] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:50.192785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c0071-aaff-42db-87fc-92980337b592 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.192785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c0071-aaff-42db-87fc-92980337b592 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.192959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c0071-aaff-42db-87fc-92980337b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.220187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a95c0071-aaff-42db-87fc-92980337b592 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:50.221711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a95c0071-aaff-42db-87fc-92980337b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:50.257362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e819cd-c24a-4ae9-aac5-c6a338460e7f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.296192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.305122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.305782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.306226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.306537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/481259c751234a5a936a8b0660eb8cac/roles/31a3f360a76e4dd9a17451573b41967e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.317893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 309/626] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:50.317893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c0071-aaff-42db-87fc-92980337b592 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.336219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=481259c751234a5a936a8b0660eb8cac, role_id=31a3f360a76e4dd9a17451573b41967e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.337662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.407101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 318/627] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:14:50 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/481259c751234a5a936a8b0660eb8cac/roles/31a3f360a76e4dd9a17451573b41967e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:14:50.408869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.439712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c0071-aaff-42db-87fc-92980337b592 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.482145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.483320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.483855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.484377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.485343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:50.486174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.488040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.503763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 310/628] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:50.510470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.516573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.553857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b874b-b350-40da-b034-635966f91fc2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.596063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af3b14-a0af-4fb6-8ef4-1c8cfc1d79f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.596890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af3b14-a0af-4fb6-8ef4-1c8cfc1d79f5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.597237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af3b14-a0af-4fb6-8ef4-1c8cfc1d79f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.597574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af3b14-a0af-4fb6-8ef4-1c8cfc1d79f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.618309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af3b14-a0af-4fb6-8ef4-1c8cfc1d79f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b6af3b14-a0af-4fb6-8ef4-1c8cfc1d79f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:50.623087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:50.624067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:50.624889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 319/629] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:50.626081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6a29-e719-4d8c-a41b-ebdabae26704 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.626081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6a29-e719-4d8c-a41b-ebdabae26704 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.626319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a6a29-e719-4d8c-a41b-ebdabae26704 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.627829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 320/630] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:50.631314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a6a29-e719-4d8c-a41b-ebdabae26704 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.632074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.695317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f3474-af1d-4b59-9ef5-11603b05e118 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.732427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d613f-e255-4975-9b5a-87414dba6905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.733082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d613f-e255-4975-9b5a-87414dba6905 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.733420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d613f-e255-4975-9b5a-87414dba6905 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.733688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d613f-e255-4975-9b5a-87414dba6905 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.743307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d613f-e255-4975-9b5a-87414dba6905 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.744900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d613f-e255-4975-9b5a-87414dba6905 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.753953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a6a29-e719-4d8c-a41b-ebdabae26704 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-522d613f-e255-4975-9b5a-87414dba6905 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:50.757938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:50.759335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:50.761000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:50.766372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 311/631] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:50.766372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecde035-d50d-404d-8ab2-c7a92bcba125 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.766372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecde035-d50d-404d-8ab2-c7a92bcba125 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.766372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecde035-d50d-404d-8ab2-c7a92bcba125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.783610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.787321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7ecde035-d50d-404d-8ab2-c7a92bcba125 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:50.788609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7ecde035-d50d-404d-8ab2-c7a92bcba125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:50.789616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.790251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.795071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.795071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.795071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:50.795071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.795071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.802987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 321/632] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:50.803817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.866117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 312/633] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:50.867141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecde035-d50d-404d-8ab2-c7a92bcba125 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:50.912158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:50.927244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849757f-22b5-4cbb-8b43-2a0484ccee28 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.964713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:50.965227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:50.965502 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:50.965769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:50.966531 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:50.966831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:50.968453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:50.979618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecde035-d50d-404d-8ab2-c7a92bcba125 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:50.988061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 322/634] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:50.989164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.027522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.029034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.029323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.029596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.043644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.045617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.046783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:51.066436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:51.067611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:51.068713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:51.068713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:51.068713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:51.075919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 313/635] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:51.077019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.104802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6fc8d-fcf8-4ffb-8a12-172928944310 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.144586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.145380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.145808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.146190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.158697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:51.163993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:51.165168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:51.166210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:51.173175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 323/636] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:51.174299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.196136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.207400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7b000-d8f9-4f14-b45d-b973efde1e48 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.252829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.253599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.253998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.254194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.294018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fae48-7139-47d8-81c7-62679afb3457 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.306685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.336949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.337900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.338578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.339084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.340010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:51.340509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.341491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.342303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.342994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.356340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 324/637] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:51.357547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.461345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.472306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 314/638] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:51.473412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.478821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd730a-5290-4c9d-93be-7ba932c21098 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.523592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.524122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.524405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.524682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.525732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:51.526043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.527721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.543204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 325/639] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:51.543204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.578888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.585297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41973e4f-0b32-4232-8e75-b77bc92499d4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.621830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.622598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.622729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.622994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.631699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.633140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:51.644739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:51.645727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:14:51.646670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:51.650535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 315/640] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:51.651468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.662464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d724e-d920-41cf-8365-4dff32645631 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.701251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf7a6a-8923-4722-a66c-77bdc98f9678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.702227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf7a6a-8923-4722-a66c-77bdc98f9678 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.702539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf7a6a-8923-4722-a66c-77bdc98f9678 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.703030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf7a6a-8923-4722-a66c-77bdc98f9678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.709099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.713805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf7a6a-8923-4722-a66c-77bdc98f9678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.715268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf7a6a-8923-4722-a66c-77bdc98f9678 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0bcf7a6a-8923-4722-a66c-77bdc98f9678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:51.731456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:51.732529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:51.733424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:51.733424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:51.733424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:51.733424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 326/641] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:51.738793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc58a3-e087-4023-a3e0-a7f579fb79a5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.739101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc58a3-e087-4023-a3e0-a7f579fb79a5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.739342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc58a3-e087-4023-a3e0-a7f579fb79a5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.740845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 327/642] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:51.741529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc58a3-e087-4023-a3e0-a7f579fb79a5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.764082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816d2f8-71c3-4da6-8e63-75defec6dc38 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.798972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.799726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.800011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.800277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.810649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.812088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:51.827290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:51.828243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:51.829200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:51.829200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:51.829200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:51.829200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 316/643] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:51.830049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:51.840711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.859062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc58a3-e087-4023-a3e0-a7f579fb79a5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.897782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a211-9bb0-4d07-9f36-664771368a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.898384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a211-9bb0-4d07-9f36-664771368a45 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.898700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a211-9bb0-4d07-9f36-664771368a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.898947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a211-9bb0-4d07-9f36-664771368a45 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.899674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a211-9bb0-4d07-9f36-664771368a45 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:51.899974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a211-9bb0-4d07-9f36-664771368a45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.901393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a211-9bb0-4d07-9f36-664771368a45 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:51.921452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 328/644] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:51.921618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f4b4f-5c6a-48c2-9145-bc2107725b9c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.921878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f4b4f-5c6a-48c2-9145-bc2107725b9c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.922150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f4b4f-5c6a-48c2-9145-bc2107725b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.941074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ef6f4b4f-5c6a-48c2-9145-bc2107725b9c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:51.942249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ef6f4b4f-5c6a-48c2-9145-bc2107725b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:51.951299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f2ccb-d7d5-499a-a0c4-fe6a007e3e62 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:51.953662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:51.988627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:51.989753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:51.990031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:51.990589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:51.991223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:51.991572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:51.993717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.010873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 317/645] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:52.010873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.023053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 329/646] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:52.035613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f4b4f-5c6a-48c2-9145-bc2107725b9c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.114120 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.120028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7f821-9308-492f-ba4d-56aa70b47255 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.144395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f4b4f-5c6a-48c2-9145-bc2107725b9c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.156533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.157247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.157649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.158029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.158722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:52.159024 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.160411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.176074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 318/647] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:52.176074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.186185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.186835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.187112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.187377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.200519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.261487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.263084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.281820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067c791-b7e9-4182-8e3c-0f41e02922ce None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.318317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.318968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.319243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.319510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.326278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.327728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.337140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:52.340627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:52.341484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 319/648] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:52.347181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.377793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 330/649] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:52.379188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.439042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.455308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f6584-5263-44ce-a3b6-ee6b13c4dfdd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.487452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df22117-c6f7-4517-8053-916c99b2070d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.496426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.497527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.497818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.498093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.512338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:52.520379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:52.521334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:52.523011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 320/650] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:52.524968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.525234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.525854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.526128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.526425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.535380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.536856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:52.548881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:52.549853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 331/651] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:52.550853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.585462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.628049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5491e4b-9f2b-4138-955e-3a42be81fbf3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.653065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b435fe-f983-425e-88ff-81f57dc6ee7e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.662273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.663187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.663556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.663916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.664856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:52.665250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.666904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.679629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 321/652] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:52.680416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.683716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.692208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422b5a7-4944-4fdc-b8d2-c3f2ec8c4e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.692818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422b5a7-4944-4fdc-b8d2-c3f2ec8c4e89 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.693091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422b5a7-4944-4fdc-b8d2-c3f2ec8c4e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.693361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422b5a7-4944-4fdc-b8d2-c3f2ec8c4e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.703485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422b5a7-4944-4fdc-b8d2-c3f2ec8c4e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.705516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422b5a7-4944-4fdc-b8d2-c3f2ec8c4e89 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a422b5a7-4944-4fdc-b8d2-c3f2ec8c4e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:52.719565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:52.721478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:52.722408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:52.722408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:52.722408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:52.722408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 332/653] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:52.722632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae29bb-a208-4fe8-a848-1ee827bee320 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.722730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae29bb-a208-4fe8-a848-1ee827bee320 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.723000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae29bb-a208-4fe8-a848-1ee827bee320 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.724223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 333/654] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:52.724923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae29bb-a208-4fe8-a848-1ee827bee320 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.785248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccd814-e586-44ae-a51f-cf6e8095b4c9 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.808638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.820395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.820986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.821263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.821533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.822187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:52.822517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.823890 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.824973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae29bb-a208-4fe8-a848-1ee827bee320 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.837283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 322/655] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:52.837283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.862131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.862879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.863224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.863503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.864228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:52.864534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.866203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.883515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 334/656] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:52.884249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:52.897797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:52.933666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818902d1-a033-4cc1-b3ad-9e7968aa9276 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:52.967425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ab7a9-b9d7-4027-95ba-324057247dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:52.968012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab7a9-b9d7-4027-95ba-324057247dde None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.968278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab7a9-b9d7-4027-95ba-324057247dde None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.968548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab7a9-b9d7-4027-95ba-324057247dde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.969260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ab7a9-b9d7-4027-95ba-324057247dde None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=c4cfb26c57514e2cbaa73cb036118dfd) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:52.969571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ab7a9-b9d7-4027-95ba-324057247dde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:52.971108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ab7a9-b9d7-4027-95ba-324057247dde None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:52.981490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 323/657] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/role_assignments?user.id=c4cfb26c57514e2cbaa73cb036118dfd&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:52.981490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d3ad-acb6-409f-b3d6-125491928cde None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:52.981490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d3ad-acb6-409f-b3d6-125491928cde None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:52.981490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d3ad-acb6-409f-b3d6-125491928cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:52.995717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122b991-c949-4976-93fb-2e59d160f9d6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.006330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ecb3d3ad-acb6-409f-b3d6-125491928cde None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:53.007601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ecb3d3ad-acb6-409f-b3d6-125491928cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:53.035828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.036617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.036994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.037367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.038143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:53.038560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.039954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.056959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 335/658] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:53.056959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.059933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.086152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 324/659] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:53.095808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d3ad-acb6-409f-b3d6-125491928cde None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.164744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.178932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad9a91-8b6f-4263-ba7d-f12fa21746ae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.216421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d3ad-acb6-409f-b3d6-125491928cde None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.221440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.222072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.222381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.222680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.234419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.236899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:53.252540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:53.253492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:53.254400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:53.254400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:53.254400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:53.254400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 336/660] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:53.254642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.259473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.260114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.260372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.260810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.318752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.332663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.334125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.367704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8d63-6715-45bb-bdb9-a07a3f5e18d0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.395706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.402723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.403343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.403625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.403885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.411276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.412701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:53.424380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:53.425244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 337/661] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:53.426132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.443996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 325/662] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:53.444985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.537241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023c5e3-9138-4f8d-af97-7e2f20492fb2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.549773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05631bb2-d0ac-47b3-970b-f30074f3230c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.551500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.571905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.572753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.573175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.573689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.584737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.585340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.585513 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:53.585621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.585873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.586538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:53.586840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.588211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:53.591779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:53.592964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:14:53.593997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:53.596970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 338/663] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:53.598131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.604746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 326/664] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:53.607408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.632765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.712426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7926fff-d848-4ebf-8a82-744a9369bc26 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.718736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c755b8-f89a-4d48-9817-88d651874c6c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.749260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.749885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.750292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.750597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.755829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.756427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.756706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.756979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.757658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:53.758187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.759407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.761765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.763241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.772584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 339/665] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:53.773415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:53.783424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:53.784394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:53.785267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:53.785267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:53.785267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:53.785267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 327/666] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:53.785502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.818748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.860647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:53.891041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f1979-809e-4b71-b2d0-5fe97ef9df71 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.896662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b2afe-6c9a-4bf5-9c34-72e9856e3f28 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:53.932601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.933229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.933509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.933779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.940220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:53.940876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:53.941161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:53.941434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:53.941533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.942097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:53.942439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:53.942953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.943779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:53.952083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 340/667] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:53.954746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:53.960663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 328/668] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:53.961685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:53.962749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.053990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.060050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3ebe9-2930-4e6f-ab85-8f9088898ad7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.068009 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc9b3a-12f1-437b-86b4-8c493c969e2a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.098237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.098866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.099140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.099408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.100121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=1149adba4d37480b8d4fee34c77d340d) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:54.100424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.100972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.101830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.107547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 341/669] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/role_assignments?user.id=1149adba4d37480b8d4fee34c77d340d&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:54.108321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.121042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.122147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.122657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.123064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.124348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:54.125188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.127279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.142773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 329/670] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:54.143978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.238700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bc0e2-b484-44f0-ac3f-3dc90bae3802 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.259256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d0b83-568f-42e5-b015-6bf32dbe0952 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.276727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.277482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.277917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.278284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.289175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.290731 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.297565 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.298401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.298849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.299266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.300163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:54.300463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.301874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:54.306774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:54.307701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:54.308605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:54.308605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:54.308605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:54.310318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 342/671] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:54.311403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.320156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 330/672] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:54.321173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.332725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.379602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.436890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a794529-d5bc-4f50-82a6-ebe5b6f53874 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.437261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21768a33-f426-4c43-b0d0-a0c445e0eeb8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.479171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.479384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.479801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.479907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.480081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.480206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.480355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.480443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.481169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:54.481473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.483335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.488357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.490263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.496876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 331/673] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:54.502459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:54.510219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:54.511131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:54.512013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:54.514443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 343/674] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:54.515274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.607134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.632407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6e8e2-0ebc-44e2-84e1-9b3b5ef2c9f1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.645425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995a5d9-392f-483a-aff6-7d1522716eb0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.646289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.681372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.682515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.682941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.683486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.689119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.689746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.690125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.690534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.698521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.700100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:54.712175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:54.713326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:14:54.714507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:54.721313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 344/675] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:54.721313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.790768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.792866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.840112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf03a1-bfbd-4b18-8f19-1c9141d77307 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:54.873451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:54.879502 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:54.880391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:54.881006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:54.881440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:54.882427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:54.882896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:54.884547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:54.898989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 332/676] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:54.899528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 345/677] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:54.904541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.911331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:54.934721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.015380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d9203-b583-4f25-bd8a-de1572f2784b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.025558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31c625-d858-4142-a70a-a14f2f8711b5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.050784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.051333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.051613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.051883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.062633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.064621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593a121-16e5-4198-a2cb-268509449fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.064751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.065561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593a121-16e5-4198-a2cb-268509449fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.066032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593a121-16e5-4198-a2cb-268509449fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.066563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593a121-16e5-4198-a2cb-268509449fd8 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.067585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593a121-16e5-4198-a2cb-268509449fd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:55.068065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593a121-16e5-4198-a2cb-268509449fd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.070141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593a121-16e5-4198-a2cb-268509449fd8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:55.079966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:55.081181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:55.082451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:55.082451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:55.082451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:55.082451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 346/678] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:55.082451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.097560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 333/679] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 02:14:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:55.097560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191111c-67c5-40f4-87ce-239b189889f8 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.097560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191111c-67c5-40f4-87ce-239b189889f8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.097560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191111c-67c5-40f4-87ce-239b189889f8 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.102202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 334/680] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:55.102990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191111c-67c5-40f4-87ce-239b189889f8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.136024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.175601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.193385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb937d6-3365-46a2-84c0-602cd277ab95 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.211884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191111c-67c5-40f4-87ce-239b189889f8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.229044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.229639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.229923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.230196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.230950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=38206a70a8b24cffabac622175335154) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:55.231259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.232682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.250386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 347/681] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/role_assignments?user.id=38206a70a8b24cffabac622175335154&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:55.250386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.264451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12aa92-373f-48bb-a3ff-ca8da921243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.265844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12aa92-373f-48bb-a3ff-ca8da921243b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.266127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12aa92-373f-48bb-a3ff-ca8da921243b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.266858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12aa92-373f-48bb-a3ff-ca8da921243b None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.267538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12aa92-373f-48bb-a3ff-ca8da921243b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:55.267846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12aa92-373f-48bb-a3ff-ca8da921243b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.269951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12aa92-373f-48bb-a3ff-ca8da921243b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.290778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 335/682] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:55.292188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5347a2-9e06-477c-a6d2-7837b7c37828 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.292471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5347a2-9e06-477c-a6d2-7837b7c37828 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.292743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5347a2-9e06-477c-a6d2-7837b7c37828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.317699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5b5347a2-9e06-477c-a6d2-7837b7c37828 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:55.319392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5b5347a2-9e06-477c-a6d2-7837b7c37828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:55.369806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6f85-c5b7-4846-874d-3fc20b6d35e0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.383014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.418065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.418628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.418911 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.419186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.423061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 336/683] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:55.424509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5347a2-9e06-477c-a6d2-7837b7c37828 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.496265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.514386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.516189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.551123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5347a2-9e06-477c-a6d2-7837b7c37828 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.585473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.586025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.586297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.586599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.593878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.595913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:55.607175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:55.608104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:55.609010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 337/684] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:55.614208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.648356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 348/685] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:14:55.649113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.661379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.724512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6665-c238-4a5b-a1ca-9bd2f91e92a3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.764057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.769558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.770744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.771516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.772476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.775324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0255a4-bd8c-4caf-834e-d1a4bd144998 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.785216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:14:55.791002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:55.792013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:14:55.792865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:55.797318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 338/686] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:55.798342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.822311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.823747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.824481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.824962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.826223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:55.827147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.831063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.853242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 349/687] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:55.854708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:55.917285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee719a-4911-475f-a723-f0d0e658f7e1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.949644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:55.966222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:55.967141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:55.967563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:55.967972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:55.978798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:55.980495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:55.989495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df222c57-f5db-4d45-9103-25890f429b2d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:55.996085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:55.997002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:55.997889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:55.997889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:55.997889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:55.999887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 339/688] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:56.001444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.028517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.029200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.029595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.029861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.030730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:56.031097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.032770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.035195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.047361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 350/689] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:56.048845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.116582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f51e6-3fab-40d6-a714-10958ed91cc4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.152690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.153650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.154145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.154637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.155573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=e8839af749e44fac8a7a311b61463c10) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:56.156153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.157919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.160321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aec2c7-744e-4bd2-a10b-30ff0a483dc1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.165019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 340/690] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/role_assignments?user.id=e8839af749e44fac8a7a311b61463c10&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:56.166050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.217567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.237916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf300-9d55-4f67-a979-dd0d701251c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.238551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf300-9d55-4f67-a979-dd0d701251c1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.238830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf300-9d55-4f67-a979-dd0d701251c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.242213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf300-9d55-4f67-a979-dd0d701251c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.242213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf300-9d55-4f67-a979-dd0d701251c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:56.242213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf300-9d55-4f67-a979-dd0d701251c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.242213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf300-9d55-4f67-a979-dd0d701251c1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.262255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 351/691] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:56.264730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ad519-c762-4049-bdd9-d302b9265ec5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.265405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ad519-c762-4049-bdd9-d302b9265ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.265971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ad519-c762-4049-bdd9-d302b9265ec5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.273138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 352/692] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:56.273138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ad519-c762-4049-bdd9-d302b9265ec5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.276208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.284203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d7ce7-da3a-47df-bc7d-744624ff43db None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.322980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb3d43-1424-4797-8f9c-7e80e8109d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.323684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3d43-1424-4797-8f9c-7e80e8109d50 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.323965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3d43-1424-4797-8f9c-7e80e8109d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.324235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3d43-1424-4797-8f9c-7e80e8109d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.336672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb3d43-1424-4797-8f9c-7e80e8109d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.338158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb3d43-1424-4797-8f9c-7e80e8109d50 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-86eb3d43-1424-4797-8f9c-7e80e8109d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:56.356748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:56.357878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:56.359093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:56.359093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:56.359093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:56.359093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 341/693] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:56.360095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246b562-905f-4d0c-a48f-cb6e62bf4f46 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.360471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246b562-905f-4d0c-a48f-cb6e62bf4f46 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.360962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246b562-905f-4d0c-a48f-cb6e62bf4f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.382238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4246b562-905f-4d0c-a48f-cb6e62bf4f46 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:56.382458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4246b562-905f-4d0c-a48f-cb6e62bf4f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:56.422009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ad519-c762-4049-bdd9-d302b9265ec5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.470376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.470376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.470376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.470376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.475078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 342/694] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:56.476045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.476161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246b562-905f-4d0c-a48f-cb6e62bf4f46 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.477816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:56.490466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:56.491569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:56.492683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:56.498376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 353/695] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:56.499707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.561338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.578490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.612279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246b562-905f-4d0c-a48f-cb6e62bf4f46 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.619248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71832053-7da5-4f82-bf85-ec421553f761 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.651086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.652103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.652591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.653049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.653886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:56.654407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.655973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.665153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.666305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.666611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.666899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.667579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:56.668151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.669569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.675533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 343/696] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:56.676637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.686005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 354/697] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:56.686946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.803429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3440b-cb04-4eaf-89fa-ad06d3b81992 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.844383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.845056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.845334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.845614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:56.846278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:56.847694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:56.851698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:56.862529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:56.863577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:14:56.864406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:56.869079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 344/698] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:56.870604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:56.939327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b762723-56a1-4fd3-96d8-87b378ef4156 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:56.954206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:56.990758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:56.992377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:56.992886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:56.993645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.000617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d0027-6b37-41e1-8242-21047f056d77 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.005289 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.007065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:14:57.023104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:57.024152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:14:57.025112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:57.028116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 355/699] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:57.029115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.050739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.051985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.052442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.052858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.053729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=481259c751234a5a936a8b0660eb8cac) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:57.054182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.056629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.071764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 345/700] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:56 2022] GET /identity/v3/role_assignments?user.id=481259c751234a5a936a8b0660eb8cac&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:57.071764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.198937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.215785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21431f22-1327-4ff7-8128-574ec9d2e465 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.263238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f04ae-3bdf-47a6-8413-46214bfaf6a1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.286643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.287719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.288297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.288943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.289782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:57.290404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.292088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.302600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 356/701] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:57.304376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.306983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.307840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.308010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.308365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.309156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:57.310094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.311001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.329864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 346/702] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:57.335296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.346579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.436578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d587b5-a88a-44ee-8005-216d0b4878ba None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.456112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e19ae5-c058-42db-87d5-421f74de7cc5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.483529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.487585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a39e2-1f6f-4327-adc3-4266ff79549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.488895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a39e2-1f6f-4327-adc3-4266ff79549b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.489512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a39e2-1f6f-4327-adc3-4266ff79549b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.490477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a39e2-1f6f-4327-adc3-4266ff79549b None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.491434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a39e2-1f6f-4327-adc3-4266ff79549b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:57.491891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a39e2-1f6f-4327-adc3-4266ff79549b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.493734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a39e2-1f6f-4327-adc3-4266ff79549b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.498284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.501360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.501360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.501360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.501579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 357/703] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:57.501579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe85f-7fc5-42c6-909c-3855b45e3049 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.502099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe85f-7fc5-42c6-909c-3855b45e3049 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.503148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe85f-7fc5-42c6-909c-3855b45e3049 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.506411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 358/704] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:57.509225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f84c2b-a727-4df3-8b8e-5e2685061be6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.509701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f84c2b-a727-4df3-8b8e-5e2685061be6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.510122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f84c2b-a727-4df3-8b8e-5e2685061be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.531435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-62f84c2b-a727-4df3-8b8e-5e2685061be6 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:57.533599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62f84c2b-a727-4df3-8b8e-5e2685061be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:57.592879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.595767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.629440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 359/705] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:57.654710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f84c2b-a727-4df3-8b8e-5e2685061be6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.684492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.778303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 347/706] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:14:57.780070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.791324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f84c2b-a727-4df3-8b8e-5e2685061be6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.807238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.830028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.830760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.831043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.831314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.845219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.847138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:57.866720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:57.867701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:57.869194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:57.869194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:57.869194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:57.869194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 360/707] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:14:57.873705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:57.902249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ec89d-5e90-4dcd-b646-0a46444491fc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.942612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:57.943649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:57.944156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:57.944660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:57.962383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:57.962383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:57.975525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:14:57.996604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:57.997748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:14:57.999097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:14:57.999097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:14:57.999097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:57.999282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5b560-cd13-4b30-9c44-a74e8de0c631 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:57.999356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 348/708] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:58.005749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:58.059062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:58.060039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.060522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.060991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.061861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:58.062379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:58.064727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:58.093770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 361/709] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:58.106535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:58.111743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:58.136369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904b49e-880b-4d67-8dc4-1164064be21f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:58.179000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:58.179746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.180092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.180365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.181015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:58.181316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:58.183033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:58.198955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 349/710] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:58.208747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:58.309032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7da970-ec2e-40a0-afd9-98d818123585 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:58.318466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372a7c0-bee7-421d-9c18-19d131aba2b3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:58.357010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:58.357640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.357965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.358240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.360898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:58.368020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:58.369779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:58.381326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d59cbc-c06a-4187-bbe1-7e9a1a3d1313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:58.382060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d59cbc-c06a-4187-bbe1-7e9a1a3d1313 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.382469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d59cbc-c06a-4187-bbe1-7e9a1a3d1313 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.382844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d59cbc-c06a-4187-bbe1-7e9a1a3d1313 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:14:58.383120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:58.384161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:14:58.385176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:14:58.385604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d59cbc-c06a-4187-bbe1-7e9a1a3d1313 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:58.386758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d59cbc-c06a-4187-bbe1-7e9a1a3d1313 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:58.387782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 350/711] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:58.397156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:58.414393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:58.508507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 362/712] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:14:58 2022] POST /identity/v3/users => generated 254 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:14:58.518406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9800e-9c13-496b-9c75-a007c79353a6 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.518984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34834a1a-c5af-47be-b8f4-8a2e698a45e4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:58.519454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9800e-9c13-496b-9c75-a007c79353a6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.519896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9800e-9c13-496b-9c75-a007c79353a6 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.521709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 363/713] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:58.528930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844f1c4-8433-4a3b-8797-fc34298c873f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.529504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844f1c4-8433-4a3b-8797-fc34298c873f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.530004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844f1c4-8433-4a3b-8797-fc34298c873f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.564271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:58.564921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.565202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.565469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.566161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:58.566534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:58.568060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:58.568580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3844f1c4-8433-4a3b-8797-fc34298c873f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:58.570071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3844f1c4-8433-4a3b-8797-fc34298c873f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:58.581503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 351/714] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:14:58.591639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:58.654137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 364/715] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:14:58.695099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:58.702254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844f1c4-8433-4a3b-8797-fc34298c873f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:58.741850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6028f-bbd7-4618-9902-6f71f09cb0db None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:58.783276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:58.784538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.784942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.785215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.785937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=c4cfb26c57514e2cbaa73cb036118dfd) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:58.786242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:58.788624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:58.795887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 352/716] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:14:58 2022] GET /identity/v3/role_assignments?user.id=c4cfb26c57514e2cbaa73cb036118dfd&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:58.839974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844f1c4-8433-4a3b-8797-fc34298c873f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:58.871377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:58.882533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:58.883179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:58.883460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:58.883734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:58.888024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:58.888024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:14:58.897494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:14:58.898596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:14:58.900207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 365/717] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:14:58 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:14:58.904545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:59.057218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9512d-c6f7-4c6b-a03c-250e9184521c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:59.077338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:59.101655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34988823-599b-4616-a5fc-ce65b81bf05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:59.102195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34988823-599b-4616-a5fc-ce65b81bf05d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:59.102505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34988823-599b-4616-a5fc-ce65b81bf05d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:59.102771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34988823-599b-4616-a5fc-ce65b81bf05d None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:59.103393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34988823-599b-4616-a5fc-ce65b81bf05d None admin] RBAC: Adding query filter params (name=network) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:59.103690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34988823-599b-4616-a5fc-ce65b81bf05d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:59.105126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34988823-599b-4616-a5fc-ce65b81bf05d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:59.115138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 353/718] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:59.120769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:14:59.262212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bbde-2753-41db-badd-41e0e10ced1a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:14:59.301890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160392e-4127-4f3c-880d-d5987d7f07dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:14:59.302922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160392e-4127-4f3c-880d-d5987d7f07dc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:59.303422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160392e-4127-4f3c-880d-d5987d7f07dc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:59.303906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160392e-4127-4f3c-880d-d5987d7f07dc None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:59.304760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160392e-4127-4f3c-880d-d5987d7f07dc None admin] RBAC: Adding query filter params (type=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:14:59.305286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160392e-4127-4f3c-880d-d5987d7f07dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:14:59.306901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160392e-4127-4f3c-880d-d5987d7f07dc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:14:59.312070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 366/719] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:14:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:14:59.849780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571de78-d72d-4869-b1ff-69d25a67c287 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:59.850336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571de78-d72d-4869-b1ff-69d25a67c287 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:59.850336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571de78-d72d-4869-b1ff-69d25a67c287 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:59.851678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 354/720] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:14:59.856836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6482-15b7-4b87-96a2-bbda8af9afef None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:14:59.857123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6482-15b7-4b87-96a2-bbda8af9afef None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:14:59.857393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6482-15b7-4b87-96a2-bbda8af9afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:14:59.884368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-731f6482-15b7-4b87-96a2-bbda8af9afef None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:14:59.885556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-731f6482-15b7-4b87-96a2-bbda8af9afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:14:59.956942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:14:59.978840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 367/721] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:15:00.013968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571de78-d72d-4869-b1ff-69d25a67c287 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.066872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142fe7e-ad55-4ab1-8d2c-0f1032b33f69 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.067242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142fe7e-ad55-4ab1-8d2c-0f1032b33f69 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.067419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142fe7e-ad55-4ab1-8d2c-0f1032b33f69 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.068738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 368/722] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:00.073869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a71b-b5a7-484b-84c0-db0ee66da765 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.074206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a71b-b5a7-484b-84c0-db0ee66da765 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.074447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a71b-b5a7-484b-84c0-db0ee66da765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.115769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d484a71b-b5a7-484b-84c0-db0ee66da765 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:00.116878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d484a71b-b5a7-484b-84c0-db0ee66da765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:00.138306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571de78-d72d-4869-b1ff-69d25a67c287 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:00.177419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:00.178294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.179185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.179474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.204533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:00.209755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:00.210779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:00.211698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 355/723] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:00.216811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.256191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 369/724] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:15:00.294397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:00.295073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a71b-b5a7-484b-84c0-db0ee66da765 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.392585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97609a8b-5762-47c4-bde0-75fabe3a0ee5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:00.458399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:00.459093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.459372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.459638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.460295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:00.460592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:00.461972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:00.470679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 356/725] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:00.475725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.491854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a71b-b5a7-484b-84c0-db0ee66da765 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:00.527469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:00.580881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4f66-da0b-4746-9422-6911b8953de0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:00.590691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:00.603660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:00.605646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.605646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.605646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.605646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:00.606849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:00.617148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:00.617830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.618112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.618401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.629951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:00.631507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:00.646486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:00.647483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:00.648453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:00.648453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:00.648453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:00.648453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 357/726] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:00.653687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.702416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 370/727] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:00 2022] POST /identity/v3/services => generated 251 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:00.773653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95360-60ab-4884-9f92-adfa9535b610 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:00.817500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:00.818117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:00.818421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:00.818698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:00.819347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:00.819646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:00.821037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:00.835413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 358/728] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:00.841006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:00.989358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544bf52-fd2e-4900-ace3-7ac6b6b865c7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:01.028231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb1036-ec7c-4c64-be6e-924873bc248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:01.028874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1036-ec7c-4c64-be6e-924873bc248d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.029152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1036-ec7c-4c64-be6e-924873bc248d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.029424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1036-ec7c-4c64-be6e-924873bc248d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.039603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb1036-ec7c-4c64-be6e-924873bc248d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:01.041054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb1036-ec7c-4c64-be6e-924873bc248d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5cfb1036-ec7c-4c64-be6e-924873bc248d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:01.054787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:01.056996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:01.059267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 371/729] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:01.066400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:01.079119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:01.175648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913073f-4dbc-49bd-a004-b1e9586d61f0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:01.179117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee59868-e803-407e-b5a3-0326a1441e93 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.179117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee59868-e803-407e-b5a3-0326a1441e93 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.179117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee59868-e803-407e-b5a3-0326a1441e93 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.180565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 372/730] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:01.186590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc72c7a-9ab9-41a3-9d98-19abf77d41f5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.186872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc72c7a-9ab9-41a3-9d98-19abf77d41f5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.187148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc72c7a-9ab9-41a3-9d98-19abf77d41f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.208962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6fc72c7a-9ab9-41a3-9d98-19abf77d41f5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:01.210489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6fc72c7a-9ab9-41a3-9d98-19abf77d41f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:01.227859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:01.228523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.228793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.229111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.229845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:01.230151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:01.231888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:01.267593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 359/731] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 02:15:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:01.275282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:01.277144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:01.368178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 373/732] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:15:01.370632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0b04-d807-4618-a925-68886161c589 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.370933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0b04-d807-4618-a925-68886161c589 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.371206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0b04-d807-4618-a925-68886161c589 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.372512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 374/733] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:01.383482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41e41d-0cae-458b-a226-41bf6a0fa05c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.383761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41e41d-0cae-458b-a226-41bf6a0fa05c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.384039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41e41d-0cae-458b-a226-41bf6a0fa05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.412050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:01.419194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1c41e41d-0cae-458b-a226-41bf6a0fa05c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:01.421751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c41e41d-0cae-458b-a226-41bf6a0fa05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:01.450341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18788243-aa76-4432-ad79-65b2fccfb98b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:01.490463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:01.491074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.491446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.491760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.496885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 375/734] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:15:01.497743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41e41d-0cae-458b-a226-41bf6a0fa05c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:01.567211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:01.568652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:01.628927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:01.635795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41e41d-0cae-458b-a226-41bf6a0fa05c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:01.655919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:01.673845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:01.674783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.675157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.675518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.687365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:01.692917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:01.693916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:15:01.694799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:01.698495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 360/735] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:01 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:15:01.698495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:01.698860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 376/736] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:15:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:01.703654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:01.817905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4b8b8-8eb8-4481-a949-784ffe3e8e93 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:01.832088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c8956-9f89-4af2-8927-06e797e32b01 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:01.855194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:01.855816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.856093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.856360 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.868284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:01.869722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:01.874932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087c282-26a8-4928-940d-c1bfb1cd3914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:01.875543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087c282-26a8-4928-940d-c1bfb1cd3914 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.876036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087c282-26a8-4928-940d-c1bfb1cd3914 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.876637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087c282-26a8-4928-940d-c1bfb1cd3914 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:01.884450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:01.885484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:01.886594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:01.886594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:01.886594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:01.886594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 377/737] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:01.887086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:01.891053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087c282-26a8-4928-940d-c1bfb1cd3914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4087c282-26a8-4928-940d-c1bfb1cd3914 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:01.895943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:01.896862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:01.897699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:01.898330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:01.900751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 361/738] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:01.901803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484742d-6070-4fcc-bec7-a50c234791c7 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:01.902043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484742d-6070-4fcc-bec7-a50c234791c7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:01.902311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484742d-6070-4fcc-bec7-a50c234791c7 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:01.903576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 362/739] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:01.904252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484742d-6070-4fcc-bec7-a50c234791c7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:01.939473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.000052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165a4e-b034-4626-be54-a9827c8e4f64 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.019644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484742d-6070-4fcc-bec7-a50c234791c7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.034374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.035025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.035305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.035577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.036270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:02.036572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.037932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.045991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 378/740] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:15:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:02.046820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.060967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cd265-f7ab-45e2-a39f-262198c614b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.061826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cd265-f7ab-45e2-a39f-262198c614b6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.062288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cd265-f7ab-45e2-a39f-262198c614b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.062871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cd265-f7ab-45e2-a39f-262198c614b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.064323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cd265-f7ab-45e2-a39f-262198c614b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:02.064820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cd265-f7ab-45e2-a39f-262198c614b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.066477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cd265-f7ab-45e2-a39f-262198c614b6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.079852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 363/741] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:02.081433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15df12-7920-4a09-bd77-c8e9800661bc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.081719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15df12-7920-4a09-bd77-c8e9800661bc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.081992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15df12-7920-4a09-bd77-c8e9800661bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.101108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fd15df12-7920-4a09-bd77-c8e9800661bc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:02.102398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fd15df12-7920-4a09-bd77-c8e9800661bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:02.121512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.156013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406512-442c-4892-970f-a00c09eb0798 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.197700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.198317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.198624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.198887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.199564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:02.199860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.201243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.214723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 379/742] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:02.216059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.299227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 364/743] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:15:02.299227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15df12-7920-4a09-bd77-c8e9800661bc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.328390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.332401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74c2ff-c46f-4dab-801a-d0cbb5aac628 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.366176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.372902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.373526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.373801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.374066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.385108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.386561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:02.400776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:02.401798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:02.402740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:02.402740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:02.402740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:02.402740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 380/744] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:02.402740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.425442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15df12-7920-4a09-bd77-c8e9800661bc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.464881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.465578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.465854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.466120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.473569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.475282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:02.487369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:02.488309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:02.489225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 365/745] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:02.489996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.521957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135edc0-3f39-4d83-b5ae-d7a0cc24a774 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.558565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.559165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.559438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.559705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.570424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.571857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.574436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:02.585779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:02.588206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:02.589182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:02.589182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:02.589182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:02.589182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 381/746] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:02.589182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.602451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e22a19-bb66-445a-8c11-152a0d2cfcca None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.628949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.639234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.639835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.640111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.640376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.641011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.642563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.695122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699c734-3a02-48ae-9dca-176986fa34e7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.722381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 366/747] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:02 2022] POST /identity/v3/services => generated 240 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:02.722381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.756234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.756846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.757124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.757676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.758889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:02.759274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.761050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.779318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 382/748] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:02.780226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.861094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448fff9-bfde-42c6-bd36-c52fdabd336d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.903858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.909616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.910230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.910555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.913637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.913637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:02.913637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.913637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.913897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37aafe-350a-4180-80ff-ef1569826517 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:02.926509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:02.933123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 367/749] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:02.936957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:02.958665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:02.958665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:02.958960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:02.959200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:02.959768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:02.960070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:02.961478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:02.980583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 383/750] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:02.980583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.055093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220a6f-8a73-4d86-8957-a1a66a5ab9cc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.097328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.097989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.098219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.098533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.108772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.111621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:03.114114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61a6df-b4ab-425b-93bc-02aa39768202 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:03.127038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:03.128212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:03.129356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:03.134384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 368/751] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:03.134384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.155578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386db48-41ee-4c3c-a9fe-b0983c8f34b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.156281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386db48-41ee-4c3c-a9fe-b0983c8f34b5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.156560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386db48-41ee-4c3c-a9fe-b0983c8f34b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.156856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386db48-41ee-4c3c-a9fe-b0983c8f34b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.157618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386db48-41ee-4c3c-a9fe-b0983c8f34b5 None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=481259c751234a5a936a8b0660eb8cac) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:03.157929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386db48-41ee-4c3c-a9fe-b0983c8f34b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.159389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386db48-41ee-4c3c-a9fe-b0983c8f34b5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:03.164756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 384/752] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:02 2022] GET /identity/v3/role_assignments?user.id=481259c751234a5a936a8b0660eb8cac&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:03.165926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ab7f-897f-4009-af0f-4e021979aa73 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.166204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ab7f-897f-4009-af0f-4e021979aa73 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.166506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ab7f-897f-4009-af0f-4e021979aa73 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.167958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 385/753] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:03.173274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ab7f-897f-4009-af0f-4e021979aa73 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.192417 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:03.228034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:03.274260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1aabb-186a-489f-88ef-b3ad99729c3d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.313812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ab7f-897f-4009-af0f-4e021979aa73 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.316420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863e885-7887-4ca9-a3b0-33c30c86d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.317172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863e885-7887-4ca9-a3b0-33c30c86d1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.317511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863e885-7887-4ca9-a3b0-33c30c86d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.318608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863e885-7887-4ca9-a3b0-33c30c86d1d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.328137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863e885-7887-4ca9-a3b0-33c30c86d1d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.331480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863e885-7887-4ca9-a3b0-33c30c86d1d9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f863e885-7887-4ca9-a3b0-33c30c86d1d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:03.348587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:03.349687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:03.350715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:03.353735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 369/754] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:15:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:03.353735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3a1f8-f45e-40c4-8516-3ae3c264b3c1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.354119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3a1f8-f45e-40c4-8516-3ae3c264b3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.358444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3a1f8-f45e-40c4-8516-3ae3c264b3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.365215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.365762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.366037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.366302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.371146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:03.371723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.373316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:03.378199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5ec3a1f8-f45e-40c4-8516-3ae3c264b3c1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:03.379652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5ec3a1f8-f45e-40c4-8516-3ae3c264b3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:03.395146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 386/755] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 02:15:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:03.396246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.462505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 370/756] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:03.466374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3a1f8-f45e-40c4-8516-3ae3c264b3c1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.492487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:03.535297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224b3d-064e-4b7b-8219-d8e0d30e6c45 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.539146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:03.586998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.587865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.588314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.589338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.590228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:03.590955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.593335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:03.607480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3a1f8-f45e-40c4-8516-3ae3c264b3c1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.618945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 387/757] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:15:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:03.620142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.651608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.652575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.653086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.653664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.731119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.732606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97d77e-5cb3-434a-bdb2-259ffdf0f09e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.733187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:03.770731 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:03.778009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.779616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.779903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.780171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.782902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.782902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:03.795117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:03.796351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:15:03.797853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:03.800680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:03.801638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 388/758] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 8 02:15:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:03.802659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.881930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 371/759] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:03.883740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:03.936285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bfb48-3c80-46e9-b5c4-1c1f48159fb0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:03.974848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:03.975809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:03.976264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:03.976750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:03.992713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:03.994569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:04.011664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:04.012667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:04.013533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:04.013533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:04.013533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:04.018589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 389/760] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:04.020112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.021472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923360a3-7069-4537-9110-7c074aeace28 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.061101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.061799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.062231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.062773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.064028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:04.064328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.065704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.071419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 372/761] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 02:15:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:04.072249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.077403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.095375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.140494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8d0e-ad39-4e61-ab3a-9aa6e5054301 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.181974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.182959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.183512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.183982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.196868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.198779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:04.216075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:04.217130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:04.218020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:04.218020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:04.218020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:04.218201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e4550-ed01-4e80-8720-3b8d15dad01f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.225707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 390/762] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:04.226972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.274679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.275288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.275567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.275894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.276538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:04.276839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.278209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.294436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 373/763] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:04.295500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.352532 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.358150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd35bc-0bb1-4556-8868-bdef1c87bad8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.404338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.405627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.405980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.406257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.407146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:04.407476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.408847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.412073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.419557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 391/764] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:04.419557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.446072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e23d2e-2cf9-4e11-b9a0-1f222bdbcf1f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.492094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.492824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.493114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.493502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.494144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:04.494476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.496504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.519476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 374/765] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:04.519476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.546027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f23d5e-d087-478f-b345-ccf91e72b4ef None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.591705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.592335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.592614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.592912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.602341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.603810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:04.618784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:04.619743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:04.620675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:04.624975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 392/766] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:04.626435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.638617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ca685-6c28-4db1-bdf1-8d2a9c4fdc0e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.657827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.676688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.677661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.678163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.678697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.679666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] RBAC: Adding query filter params (interface=public, service_id=0aa5a04eeac045e9a15b9f31b698ef68, region_id=RegionOne) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:04.680176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.681742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.683874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.696689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 375/767] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/endpoints?service_id=0aa5a04eeac045e9a15b9f31b698ef68&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:04.697866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.753979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c9dec-587a-4510-9abc-b7164fa29221 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.793681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.794718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.795200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.795669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.803821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.805735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:04.819218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:04.820193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:04.821099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:04.827628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391ffc2-55a4-44fa-ac8e-7e61b85e0e95 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.828330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 393/768] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:04.829947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.867902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.869358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.869657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.869932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.871418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:04.871418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.872389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.894439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 376/769] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:04.895722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:04.929965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.951999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ada215-0d88-4ae8-a999-f6a4623da629 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:04.975649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:04.989954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:04.990591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:04.990906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:04.991180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:04.991775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:04.993287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:04.999163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 394/770] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:05.000225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:05.035028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08529d-57a2-45c3-898f-eaa51fe1d433 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:05.083011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512032c-6d90-47f7-b365-2127dc957a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:05.084645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512032c-6d90-47f7-b365-2127dc957a59 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.085187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512032c-6d90-47f7-b365-2127dc957a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.085600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512032c-6d90-47f7-b365-2127dc957a59 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.086757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512032c-6d90-47f7-b365-2127dc957a59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:05.087748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512032c-6d90-47f7-b365-2127dc957a59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:05.089790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512032c-6d90-47f7-b365-2127dc957a59 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:05.107403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 377/771] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:05.108941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16314be5-5fe0-4257-89ac-05128f8b4524 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.109360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16314be5-5fe0-4257-89ac-05128f8b4524 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.109752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16314be5-5fe0-4257-89ac-05128f8b4524 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.116362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 378/772] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:05.116362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16314be5-5fe0-4257-89ac-05128f8b4524 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:05.144203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8f5eb-fbe2-420a-9a58-c5b7d1476835 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:05.193215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd73f51-c962-4f5f-b0b0-01b439413147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:05.193941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd73f51-c962-4f5f-b0b0-01b439413147 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.194218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd73f51-c962-4f5f-b0b0-01b439413147 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.194551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd73f51-c962-4f5f-b0b0-01b439413147 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/c4cfb26c57514e2cbaa73cb036118dfd/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.196635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd73f51-c962-4f5f-b0b0-01b439413147 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=c4cfb26c57514e2cbaa73cb036118dfd, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:05.197977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd73f51-c962-4f5f-b0b0-01b439413147 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:05.243528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16314be5-5fe0-4257-89ac-05128f8b4524 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:05.262326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:05.280168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed191b4-b1ec-4aa5-bffd-cb85bf73cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:05.280811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed191b4-b1ec-4aa5-bffd-cb85bf73cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.281092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed191b4-b1ec-4aa5-bffd-cb85bf73cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.281360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed191b4-b1ec-4aa5-bffd-cb85bf73cb9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.282114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed191b4-b1ec-4aa5-bffd-cb85bf73cb9a None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=6aa66d4b5bad476183509a4e00c56967) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:05.282482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed191b4-b1ec-4aa5-bffd-cb85bf73cb9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:05.283879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed191b4-b1ec-4aa5-bffd-cb85bf73cb9a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:05.294369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 395/773] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:04 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/c4cfb26c57514e2cbaa73cb036118dfd/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:15:05.296657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba40a-8467-4cf5-bb27-56f881d86f90 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.297173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba40a-8467-4cf5-bb27-56f881d86f90 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.297918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 379/774] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:05 2022] GET /identity/v3/role_assignments?user.id=6aa66d4b5bad476183509a4e00c56967&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:05.298464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba40a-8467-4cf5-bb27-56f881d86f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.380680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:05.392289 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-70eba40a-8467-4cf5-bb27-56f881d86f90 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:05.394075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-70eba40a-8467-4cf5-bb27-56f881d86f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:05.395789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06dcd-00da-4b68-b7c1-64b37e917166 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.396076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06dcd-00da-4b68-b7c1-64b37e917166 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.396351 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06dcd-00da-4b68-b7c1-64b37e917166 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.397810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 380/775] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:05.403790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adc680-666b-460d-8329-05459779eb06 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.404083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adc680-666b-460d-8329-05459779eb06 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.404340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adc680-666b-460d-8329-05459779eb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.436307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b8adc680-666b-460d-8329-05459779eb06 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:05.439505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b8adc680-666b-460d-8329-05459779eb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:05.519334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 396/776] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:05.548859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba40a-8467-4cf5-bb27-56f881d86f90 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:05.566376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 381/777] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:05.583582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adc680-666b-460d-8329-05459779eb06 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:05.676255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba40a-8467-4cf5-bb27-56f881d86f90 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:05.740071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:05.740702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.740986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.741253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.741840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:05.743236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:05.751235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:05.759069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:05.760064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:15:05.760902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:05.764683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 397/778] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:05 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:05.768940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:05.778277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adc680-666b-460d-8329-05459779eb06 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:05.802284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:05.830802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:05.832097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:05.833002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:05.833495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:05.844858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:05.849905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:05.850937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:05.851759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:05.854301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 382/779] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:05.860934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:05.970167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c991c-3adf-423b-a6bb-36849b288031 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:05.984033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d3c79-acea-4054-85de-83be6c2951bd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:06.021119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.022007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.022515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.022929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.023730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:06.024188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.025710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.035400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 383/780] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:06.040863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:06.050669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.051514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.051959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.052372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.053143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] RBAC: Adding query filter params (name=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:06.057612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.059247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.068815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 398/781] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:06.074302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:06.118709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:06.199896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:06.236640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d54e8-1718-45f8-b626-77df2ab63237 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:06.266787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1eb9bc-495b-43fd-9982-8a88eeb07cbf None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:06.305230 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.306194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.306697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.307668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.308650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] RBAC: Adding query filter params (type=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:06.309183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.311084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.318836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 399/782] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:06.321535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.323006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.323591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.324744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:06.326073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.348956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.351646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:06.372077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:06.373061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:06.373972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:06.373972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:06.373972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:06.376646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 384/783] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:06.381743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:06.520799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:06.527752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfd458-05b7-45a1-a6f8-80ce00a06a5c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:06.552481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b9d19-02ca-4ecc-87c6-d884542b5853 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:06.590418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.591439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.591930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.592434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.593756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:06.594831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.596571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.610326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 385/784] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:06.619314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:06.622696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:06.635959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.636567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.640012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.640012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.640012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] RBAC: Adding query filter params (interface=public, service_id=fbfe978736974b7ea933048e582172a1, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:06.640012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.640012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.652863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 400/785] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:15:06 2022] GET /identity/v3/endpoints?service_id=fbfe978736974b7ea933048e582172a1&interface=public®ion_id=RegionOne => generated 194 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:06.659488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:06.804381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009097aa-0967-43d7-83b4-72e233f4a13b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:06.838675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c6e1f-52af-410b-bbf1-21b71e6df535 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:06.869914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.870492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.870774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.871042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] PATH_INFO: `/v3/users/swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.879387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.880845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.887855 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:06.888706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:06.889203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:06.889666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:06.890499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:06.892755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:06.894461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:06.895361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:06.896687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 386/786] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:06.901419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:06.902303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 401/787] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:06 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:06.914753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.012495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea3790-c99a-45e6-9797-a9054a1850ff None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.048697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.049316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.049597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.049870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.050648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:07.050957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.052404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.073312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 387/788] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 02:15:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:07.079467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.151728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.206508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaac99a-ac05-44ed-b513-6820ed7cfe38 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.241455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42b2e2-4c10-409b-aca6-2e397ee3a721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.242089 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42b2e2-4c10-409b-aca6-2e397ee3a721 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.242383 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42b2e2-4c10-409b-aca6-2e397ee3a721 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.242664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42b2e2-4c10-409b-aca6-2e397ee3a721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.254435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42b2e2-4c10-409b-aca6-2e397ee3a721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.255888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42b2e2-4c10-409b-aca6-2e397ee3a721 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8b42b2e2-4c10-409b-aca6-2e397ee3a721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:07.270979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:07.272034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:07.273494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:07.273494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:07.273494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:07.273494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 402/789] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:07.277874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.340006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de59eb-4b52-432b-8e54-8cf7761ec57b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.340324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de59eb-4b52-432b-8e54-8cf7761ec57b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.340558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de59eb-4b52-432b-8e54-8cf7761ec57b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.342154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 403/790] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:07.351228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed34a3-393b-401c-a61e-9ec0312aa8e4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.351613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed34a3-393b-401c-a61e-9ec0312aa8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.351990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed34a3-393b-401c-a61e-9ec0312aa8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.353331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.371631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b8ed34a3-393b-401c-a61e-9ec0312aa8e4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:07.373264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b8ed34a3-393b-401c-a61e-9ec0312aa8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:07.412519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315311d4-1715-4d69-9ffa-a420c3218cf6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.453291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.454305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.454803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.455283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.457555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:07.458066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.459670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.485689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 404/792] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:07.486066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 388/792] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:07.487544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.512475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed34a3-393b-401c-a61e-9ec0312aa8e4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.595224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.601991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fa1e7-550d-42bd-8436-f128ed082920 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.627154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.645612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c6ad6-64fe-4693-b56f-9a02fd85b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.646650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6ad6-64fe-4693-b56f-9a02fd85b188 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.647073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6ad6-64fe-4693-b56f-9a02fd85b188 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.647518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6ad6-64fe-4693-b56f-9a02fd85b188 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.657822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c6ad6-64fe-4693-b56f-9a02fd85b188 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.659969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c6ad6-64fe-4693-b56f-9a02fd85b188 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.675109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed34a3-393b-401c-a61e-9ec0312aa8e4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d73c6ad6-64fe-4693-b56f-9a02fd85b188 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:07.675927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:07.676992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:07.678110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:07.681242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 389/793] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:07.682714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ac76b-563e-4e29-b2fa-291de422a97e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.683143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ac76b-563e-4e29-b2fa-291de422a97e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.683554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ac76b-563e-4e29-b2fa-291de422a97e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.685283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 390/794] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:07.686851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ac76b-563e-4e29-b2fa-291de422a97e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.725732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76304f-5775-4439-9540-cfa6be5e389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.726652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76304f-5775-4439-9540-cfa6be5e389c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.727335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76304f-5775-4439-9540-cfa6be5e389c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.727789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76304f-5775-4439-9540-cfa6be5e389c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.729441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76304f-5775-4439-9540-cfa6be5e389c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.731045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76304f-5775-4439-9540-cfa6be5e389c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0a76304f-5775-4439-9540-cfa6be5e389c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:07.743252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:07.744474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:15:07.745395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:07.748293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 405/795] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:07.749688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592875-971b-4872-bc4c-e531d1fd3dfc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.750099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592875-971b-4872-bc4c-e531d1fd3dfc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.751060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592875-971b-4872-bc4c-e531d1fd3dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.772019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-91592875-971b-4872-bc4c-e531d1fd3dfc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:07.773198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-91592875-971b-4872-bc4c-e531d1fd3dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:07.805485 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ac76b-563e-4e29-b2fa-291de422a97e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.840692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d3615-2706-4650-b94f-abefea799441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:07.841288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d3615-2706-4650-b94f-abefea799441 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.841560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d3615-2706-4650-b94f-abefea799441 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.841821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d3615-2706-4650-b94f-abefea799441 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.842524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d3615-2706-4650-b94f-abefea799441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:07.842825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d3615-2706-4650-b94f-abefea799441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:07.844175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 406/796] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:07.844839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91592875-971b-4872-bc4c-e531d1fd3dfc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.854720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d3615-2706-4650-b94f-abefea799441 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:07.877026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 391/797] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:07.878099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ca8f-e180-47fb-ba24-62bcf4840d2b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:07.878451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ca8f-e180-47fb-ba24-62bcf4840d2b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:07.878714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ca8f-e180-47fb-ba24-62bcf4840d2b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:07.880050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 392/798] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:07.880553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436ca8f-e180-47fb-ba24-62bcf4840d2b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:07.895349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.935297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:07.973543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91592875-971b-4872-bc4c-e531d1fd3dfc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:07.997642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436ca8f-e180-47fb-ba24-62bcf4840d2b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.010964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.011563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.011853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.012110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.012728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:08.013025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:08.014389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:08.023966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 407/799] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:08.023966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.039370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48e295-eacb-4937-ad05-9438e2c9616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.039999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e295-eacb-4937-ad05-9438e2c9616c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.040282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e295-eacb-4937-ad05-9438e2c9616c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.040560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e295-eacb-4937-ad05-9438e2c9616c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.050923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48e295-eacb-4937-ad05-9438e2c9616c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6b48e295-eacb-4937-ad05-9438e2c9616c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:08.055578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:08.056480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:08.057346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 393/800] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:15:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:08.058779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e663-3833-4e8b-965c-837c7b9edec0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.058779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e663-3833-4e8b-965c-837c7b9edec0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.058779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e663-3833-4e8b-965c-837c7b9edec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.077226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0241e663-3833-4e8b-965c-837c7b9edec0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:08.079537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0241e663-3833-4e8b-965c-837c7b9edec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:08.125345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:08.131891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee86150-b231-4195-a287-f301116d33c9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.153244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 394/801] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:08.163246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241e663-3833-4e8b-965c-837c7b9edec0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.166385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.167043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.167959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.167959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/481259c751234a5a936a8b0660eb8cac/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.169449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=481259c751234a5a936a8b0660eb8cac, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:08.170907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:08.201520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:08.250949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 408/802] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:08 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/481259c751234a5a936a8b0660eb8cac/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:15:08.250949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.287220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241e663-3833-4e8b-965c-837c7b9edec0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.336755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.337547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.337918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.338281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.339048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:08.339444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:08.341025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:08.348422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 395/803] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 02:15:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:08.349477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.387311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b238774-6ddc-4314-b92e-f874cd8b4104 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.430224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.430938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.431214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.431482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.432173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:08.432479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:08.433870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:08.443948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 409/804] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:15:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:08.444906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.465656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:08.483497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87650e92-eea5-4674-955f-8cc16a79c6d0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.502594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:08.527240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.527870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.528200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.528778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.542669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:08.548831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:08.551953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:08.552789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:08.553421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0a566-a662-465b-8ce0-399361b2f741 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.553496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 396/805] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:08.553496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.591515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.592267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.592662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.593085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.593972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:08.595761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:08.667354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6eeef-b568-4a26-843f-392c309052ad None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.706646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 410/806] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:08.710762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.716076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.716643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.717876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.717876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.741585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:08.744628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:08.775826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:08.776844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:08.778270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:08.778270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:08.778270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:08.778270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 397/807] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:08.784959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:08.800389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:08.871239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:08.929825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7257aa-764d-4396-9f70-9e1cc77625f6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.967432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:08.967956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:08.968233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:08.969485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:08.970369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:08.970700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:08.972065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:08.972688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437f8e-0d7b-49a3-aac9-40dee1ed4903 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:08.981097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 411/808] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:08.988550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.021815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.022966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.023397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.023991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.025098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:09.025679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.027360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.043034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 398/809] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:09.049750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.107254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a369369-273a-4e04-82e0-f4c5a93019bd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.117978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.147324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.148057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.148434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.148897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.152540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.156481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6f91e-ea8a-4902-8b06-6701986f8553 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.163387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.165190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:09.183801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:09.184775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:09.185652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:09.185652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:09.185652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:09.188575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 412/810] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:09.195317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.206521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.210762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.211411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.211908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.227642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.230122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:09.257118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:09.258113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:09.259084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:09.266673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 399/811] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:09.271106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.401496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465877d-5330-4d3b-aa9c-729a3024ed0c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.435404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd6036-97f3-42e5-804c-ed36953288ac None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.445667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee40f1-a233-425a-b907-dc7555e4c150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.446273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee40f1-a233-425a-b907-dc7555e4c150 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.446577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee40f1-a233-425a-b907-dc7555e4c150 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.446846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee40f1-a233-425a-b907-dc7555e4c150 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.448070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee40f1-a233-425a-b907-dc7555e4c150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:09.448652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee40f1-a233-425a-b907-dc7555e4c150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.450279 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee40f1-a233-425a-b907-dc7555e4c150 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.463611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.467919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 413/812] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:09.470847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722d6cd-84ec-4719-bd7f-ac25e1c9789b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.471134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722d6cd-84ec-4719-bd7f-ac25e1c9789b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.471409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722d6cd-84ec-4719-bd7f-ac25e1c9789b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.472959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 414/813] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:09.473730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0bfb5-9832-49ea-b977-083ebe830e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.474336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0bfb5-9832-49ea-b977-083ebe830e5f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.474641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0bfb5-9832-49ea-b977-083ebe830e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.474912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0bfb5-9832-49ea-b977-083ebe830e5f None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.475816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0bfb5-9832-49ea-b977-083ebe830e5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:09.476121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0bfb5-9832-49ea-b977-083ebe830e5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.477917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0bfb5-9832-49ea-b977-083ebe830e5f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.478027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722d6cd-84ec-4719-bd7f-ac25e1c9789b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.501994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 400/814] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:09.503361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f35f1-1ed0-4e82-aa6b-b6e78ec9aacd None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.503645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f35f1-1ed0-4e82-aa6b-b6e78ec9aacd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.503914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f35f1-1ed0-4e82-aa6b-b6e78ec9aacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.531349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-bc2f35f1-1ed0-4e82-aa6b-b6e78ec9aacd None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:09.533648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bc2f35f1-1ed0-4e82-aa6b-b6e78ec9aacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:09.566765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.611242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722d6cd-84ec-4719-bd7f-ac25e1c9789b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.619951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 401/815] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:09.620838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f35f1-1ed0-4e82-aa6b-b6e78ec9aacd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.673502 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.674673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.674961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.675236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.686935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.689090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:09.704389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:09.705380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:09.706394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:09.707381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 415/816] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:09.708083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.742618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f35f1-1ed0-4e82-aa6b-b6e78ec9aacd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.758694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.780696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.781400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.782321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.782611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.793182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.794837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:09.817102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:09.818118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:09.819085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:09.819085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:09.819085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:09.820095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 402/817] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:09.821103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.846286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299cb79-dd13-40e0-939b-da8c164e2a39 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.856673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:09.882658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.883309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.883586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.883853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.884435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.885782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:09.895839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:09.896857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:15:09.897682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:09.901949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 416/818] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:09.902818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:09.945471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74adab-88f4-4eb2-9426-0573eacb2e29 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:09.982948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:09.983714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:09.984091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:09.984522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:09.986077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:09.986447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:09.987981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.009251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 403/819] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:10.010035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.034826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676cd75-70cb-403e-9bb1-44fb904cc4cc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.038671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.070504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.071347 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.071769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.072181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.072987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:10.073487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.074998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.087965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 417/820] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:10.089093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.121002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.126375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc24d9-4ce6-4481-be1e-7992ae2ebdad None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.162790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.163364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.163653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.163990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.164684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:10.165050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.166513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.177439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 404/821] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:10.178607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.198086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa503ac-1cf4-4efb-a934-fa56742962a4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.236624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.237605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.238060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.238577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.280830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.294804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ea951-1ab4-4793-80b2-504aa93000e2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.333884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.335354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.339357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.339986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.340262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.340531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.351009 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.353454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.369999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:10.371245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:10.372099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:10.372991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:10.375768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 405/822] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:10.376769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.451941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 418/823] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:10.453030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.504225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12ad1d-670a-4c46-9ac5-28b60cbabcc5 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.539685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c0387-75b9-4ea0-8a7a-ef083ff8577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.540503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c0387-75b9-4ea0-8a7a-ef083ff8577c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.540927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c0387-75b9-4ea0-8a7a-ef083ff8577c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.541355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c0387-75b9-4ea0-8a7a-ef083ff8577c None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.541986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c0387-75b9-4ea0-8a7a-ef083ff8577c None admin] RBAC: Adding query filter params (type=network) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:10.542289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c0387-75b9-4ea0-8a7a-ef083ff8577c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.543680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c0387-75b9-4ea0-8a7a-ef083ff8577c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.549843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 406/824] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:10.551108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c888-1b61-4fe7-81a1-f4d23654c89d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.551394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c888-1b61-4fe7-81a1-f4d23654c89d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.551663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c888-1b61-4fe7-81a1-f4d23654c89d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.554478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 407/825] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:10.554478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16c888-1b61-4fe7-81a1-f4d23654c89d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.566269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef890f-336e-479a-94e0-9bafa8c05978 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.573214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.597820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.602924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.603526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.603809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.604069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.604756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:10.605057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.606542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.619788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 419/826] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:10.620945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.675630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16c888-1b61-4fe7-81a1-f4d23654c89d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.712367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a031681-d393-43ec-848e-ce8eda2a21db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.713480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031681-d393-43ec-848e-ce8eda2a21db None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.714076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031681-d393-43ec-848e-ce8eda2a21db None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.714611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031681-d393-43ec-848e-ce8eda2a21db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.726950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a031681-d393-43ec-848e-ce8eda2a21db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.727975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab243181-9601-4c97-9c83-6009153b020a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:10.728513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a031681-d393-43ec-848e-ce8eda2a21db None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9a031681-d393-43ec-848e-ce8eda2a21db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:10.744095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:10.745052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:10.745910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:10.745910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:10.745910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:10.749785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 408/827] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:10.751812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b73e5-1447-466c-adf4-bdf64271257c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.752090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b73e5-1447-466c-adf4-bdf64271257c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.752356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b73e5-1447-466c-adf4-bdf64271257c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.765621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:10.766838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:10.767291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:10.767562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:10.768240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:10.769570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:10.772457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b56b73e5-1447-466c-adf4-bdf64271257c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:10.773834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b56b73e5-1447-466c-adf4-bdf64271257c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:10.834695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.842752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 409/828] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:10.843523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b73e5-1447-466c-adf4-bdf64271257c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.870379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 420/829] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:10.870379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:10.932719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:10.994090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b73e5-1447-466c-adf4-bdf64271257c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.013997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07729ca8-33d7-4564-805b-60bd71813d4a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.045131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.045804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.046083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.046380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/6aa66d4b5bad476183509a4e00c56967/roles/31a3f360a76e4dd9a17451573b41967e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.060896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.061505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.061783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.062054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.063344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:11.063636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.064995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.068409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=6aa66d4b5bad476183509a4e00c56967, role_id=31a3f360a76e4dd9a17451573b41967e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.069907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.082343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 421/830] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:11.083618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.130738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 410/831] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:10 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/6aa66d4b5bad476183509a4e00c56967/roles/31a3f360a76e4dd9a17451573b41967e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:15:11.133710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.188883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:11.203338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6ef86-9fee-473b-9625-8b099e7cfcdc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.206885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:11.243282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb063ff-91a3-42db-ba24-67d96a90ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.244103 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb063ff-91a3-42db-ba24-67d96a90ce3e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.244676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb063ff-91a3-42db-ba24-67d96a90ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.244676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb063ff-91a3-42db-ba24-67d96a90ce3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.251454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f365870-f5d9-4847-b877-b37416b5b79e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.259031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb063ff-91a3-42db-ba24-67d96a90ce3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cbb063ff-91a3-42db-ba24-67d96a90ce3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:11.263935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:11.264894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:11.266411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 422/832] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:11.267148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47fc78-d9a8-4d6a-9980-b330f3ae136a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.267225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47fc78-d9a8-4d6a-9980-b330f3ae136a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.267698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47fc78-d9a8-4d6a-9980-b330f3ae136a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.268973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 423/833] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:11.273860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47fc78-d9a8-4d6a-9980-b330f3ae136a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.302503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65f0eb-fe8c-4dab-b62f-fe7250e3a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.303148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f0eb-fe8c-4dab-b62f-fe7250e3a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.303422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f0eb-fe8c-4dab-b62f-fe7250e3a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.303704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65f0eb-fe8c-4dab-b62f-fe7250e3a0c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.311324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65f0eb-fe8c-4dab-b62f-fe7250e3a0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.313427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65f0eb-fe8c-4dab-b62f-fe7250e3a0c0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-fb65f0eb-fe8c-4dab-b62f-fe7250e3a0c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:11.325175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:11.326087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:11.327664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 411/834] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:11.328155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86205167-e90b-44b7-9a87-b78f99e45c5e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.328155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86205167-e90b-44b7-9a87-b78f99e45c5e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.329220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86205167-e90b-44b7-9a87-b78f99e45c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.346976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-86205167-e90b-44b7-9a87-b78f99e45c5e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:11.348160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-86205167-e90b-44b7-9a87-b78f99e45c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:11.398333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47fc78-d9a8-4d6a-9980-b330f3ae136a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.434375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 412/835] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:11.434375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86205167-e90b-44b7-9a87-b78f99e45c5e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.447691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.448548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.448888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.449331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.450109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:11.450760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.452011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.465823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 424/836] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:11.465823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.473194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:11.493162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:11.546400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86205167-e90b-44b7-9a87-b78f99e45c5e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.575772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a05e4-0957-4936-84a1-b4a8e832d638 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.582372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.582992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.583275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.583546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.584239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:11.584544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.585983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.599072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 413/837] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:11.600412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.612220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.612838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.613112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.613375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.613948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.615584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:11.624740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:11.626616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:11.627444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 425/838] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:11.628233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.713526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:11.720243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e724b-dda3-4a33-95b0-37ac2cd92878 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.726573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:11.735766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bfe24-d2b6-4e1c-b814-bdd59f1ee828 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.755981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.756635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.756912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.757182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.768466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.769907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.772301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.773193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.773470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.773738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.774490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=c4cfb26c57514e2cbaa73cb036118dfd) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:11.774793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.776179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.790385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 426/839] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/role_assignments?user.id=c4cfb26c57514e2cbaa73cb036118dfd&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:11.790385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:11.790657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:11.791698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:11.792796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:11.792796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:11.792796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:11.792796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 414/840] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:11.792796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:11.931073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0530a-e783-4548-9894-bdb4774c7953 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.939666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2940d-86c1-4920-adec-3c53d0bde60f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:11.976436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.977426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.977866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.978300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.979105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:11.979536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.980188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:11.980795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:11.981072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:11.981341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:11.981652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.981966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:11.982265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:11.985083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:11.991744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 427/841] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:11.997234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 415/842] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:11.997362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.003778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.016487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.020724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.126122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ba75-122f-4f8e-aa97-4a72bfb5baa8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:12.128606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9876-be07-4181-aadf-9d231cd02293 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:12.167405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:12.168413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.168689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.168956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.169580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:12.169875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:12.171266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:12.174407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:12.175268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.175744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.176230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.176806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 428/843] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:12.184691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:12.186588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:12.201012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:12.202028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:12.203032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 416/844] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:12.208204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.310238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.331250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b3f60-414d-4e68-985c-972dbdf367f6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:12.366946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:12.367543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.367817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.368084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.368808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:12.369104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:12.370544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:12.389399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 429/845] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:12.395177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.462945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.499757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eeeee-909f-4fb2-826f-72450e06eb99 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:12.538314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:12.539327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.539814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.540277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.610422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:12.612634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:12.679019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.717973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 417/846] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:15:12.723948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:12.829632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953a0f8-3684-4ebe-b350-d36180bcd766 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:12.865557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:12.866096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:12.866393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:12.866673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:12.875128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:12.877253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:12.878719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:12.894235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:12.895199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:12.896485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:12.896485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:12.896485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:12.896485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 430/847] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:12.901366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.006425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2388bc8-203a-4a40-877d-a91fbb98fbb8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.042582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137adbf-4ff9-4b39-8f03-cd3106ee7234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.043181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137adbf-4ff9-4b39-8f03-cd3106ee7234 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.043514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137adbf-4ff9-4b39-8f03-cd3106ee7234 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.043788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137adbf-4ff9-4b39-8f03-cd3106ee7234 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.044424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137adbf-4ff9-4b39-8f03-cd3106ee7234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:13.044766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137adbf-4ff9-4b39-8f03-cd3106ee7234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.046184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137adbf-4ff9-4b39-8f03-cd3106ee7234 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.058318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 418/848] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:13.064481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.170868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b932d-2f2d-47b1-a242-84f27a829961 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.171159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b932d-2f2d-47b1-a242-84f27a829961 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.171343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b932d-2f2d-47b1-a242-84f27a829961 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.172821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3d0fd-fda6-48d1-bbce-d08059fc1bea None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.173205 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 419/849] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:13.179861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4280d8a-1e50-4e41-855e-359fd3f4bb10 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.180252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4280d8a-1e50-4e41-855e-359fd3f4bb10 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.180679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4280d8a-1e50-4e41-855e-359fd3f4bb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.203841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a4280d8a-1e50-4e41-855e-359fd3f4bb10 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:13.205627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a4280d8a-1e50-4e41-855e-359fd3f4bb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:13.212468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.213090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.213377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.213649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.220621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.222050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:13.233528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:13.234490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:13.235507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 431/850] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:13.236475 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:13.241562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.296857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 420/851] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:13.308328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:13.316603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4280d8a-1e50-4e41-855e-359fd3f4bb10 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.366047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc768183-6cfc-4eb6-9dd4-54eefabbdb46 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.405677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.407321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.407812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.408285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.409331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:13.409861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.411670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.426321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 432/852] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:13.432200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.447085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4280d8a-1e50-4e41-855e-359fd3f4bb10 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.495731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.496962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.497681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.498131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.515217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:13.521934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:13.522936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:15:13.523782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:13.526654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 421/853] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:13.532759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.552677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:13.555059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdf73c-dc0e-4599-bf86-a1a04ac99be1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.573784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:13.602748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.603902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.604636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.605119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.606589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=481259c751234a5a936a8b0660eb8cac) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:13.607371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.610684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.626991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 433/854] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/role_assignments?user.id=481259c751234a5a936a8b0660eb8cac&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:13.652345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8a8cb-5474-46dc-8605-47a45acbd94b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.692802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.693342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.693623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.693894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.694606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:13.695090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.696588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.706946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 422/855] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:13.712786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.826867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c6a71-6206-45a0-8393-384a8fead645 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:13.864178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29fdda-8bb6-4aac-a7fa-0d13605b7617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:13.865079 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29fdda-8bb6-4aac-a7fa-0d13605b7617 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:13.865334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29fdda-8bb6-4aac-a7fa-0d13605b7617 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:13.865607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29fdda-8bb6-4aac-a7fa-0d13605b7617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:13.877408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29fdda-8bb6-4aac-a7fa-0d13605b7617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:13.878914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29fdda-8bb6-4aac-a7fa-0d13605b7617 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1d29fdda-8bb6-4aac-a7fa-0d13605b7617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:13.893961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:13.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:13.895856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:13.895856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:13.895856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:13.895856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 434/856] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:13.901287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:13.988795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.006928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f238510-e2ec-42a0-9a28-11f06a4bae3f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.007220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f238510-e2ec-42a0-9a28-11f06a4bae3f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.007482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f238510-e2ec-42a0-9a28-11f06a4bae3f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.009015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 435/857] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:14.014538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71707b9b-917c-4c28-9a8c-fe8b9b0f506e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.015099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71707b9b-917c-4c28-9a8c-fe8b9b0f506e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.015302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71707b9b-917c-4c28-9a8c-fe8b9b0f506e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.018754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8be6bd-7a9f-4c46-aaef-5114c3ee0b2d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.034990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-71707b9b-917c-4c28-9a8c-fe8b9b0f506e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:14.036669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-71707b9b-917c-4c28-9a8c-fe8b9b0f506e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:14.059735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.060827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.061346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.061820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.063240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:14.063811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.064311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.066724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.080988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 423/858] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:14.088821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.118484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 436/859] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:14.134910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71707b9b-917c-4c28-9a8c-fe8b9b0f506e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.198116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dab07-8347-492c-9207-8895ae29e11e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.236883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.237896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.238515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.239174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.242164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71707b9b-917c-4c28-9a8c-fe8b9b0f506e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.247733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.249751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.252171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:14.264518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:14.265486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:14.267003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:14.274042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 424/860] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:15:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:14.275310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.284645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.285292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.285573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.285845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.286466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.288327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:14.300065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:14.301489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:14.302921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 437/861] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 8 02:15:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:14.307222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.336488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.399706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5d15-03a7-4454-9829-3034baac9134 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.417962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c681-50ce-452b-a8b7-bc2e9aee9890 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.436710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd766e1-dbfb-4ea7-94d9-06f5885f8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.437312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd766e1-dbfb-4ea7-94d9-06f5885f8a5e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.437755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd766e1-dbfb-4ea7-94d9-06f5885f8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.438235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd766e1-dbfb-4ea7-94d9-06f5885f8a5e None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.439576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd766e1-dbfb-4ea7-94d9-06f5885f8a5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:14.440070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd766e1-dbfb-4ea7-94d9-06f5885f8a5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.441503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd766e1-dbfb-4ea7-94d9-06f5885f8a5e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.452543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.453669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.453942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.454239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.454938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:14.455238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.456587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.459900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 425/862] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 02:15:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:14.461341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837beef-12f3-430c-8d42-456c71557c16 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.461802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837beef-12f3-430c-8d42-456c71557c16 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.462234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837beef-12f3-430c-8d42-456c71557c16 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.466149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 438/863] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:14.467080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.467775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 426/864] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:14.468462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837beef-12f3-430c-8d42-456c71557c16 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.530326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.559878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.575043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac08e4-73a4-456c-bde2-e8bce108b4f3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.579393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837beef-12f3-430c-8d42-456c71557c16 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.609822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.610706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.611123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.611535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.614598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd97d0-88ee-4882-b4a0-4229a2bd2504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.615641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd97d0-88ee-4882-b4a0-4229a2bd2504 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.615641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd97d0-88ee-4882-b4a0-4229a2bd2504 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.615828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd97d0-88ee-4882-b4a0-4229a2bd2504 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.616326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd97d0-88ee-4882-b4a0-4229a2bd2504 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:14.616626 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd97d0-88ee-4882-b4a0-4229a2bd2504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.617986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd97d0-88ee-4882-b4a0-4229a2bd2504 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.622553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 427/865] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:14.623675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c13c9-a9e2-4fb2-8c85-280518c4aac8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.623952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c13c9-a9e2-4fb2-8c85-280518c4aac8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.624223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c13c9-a9e2-4fb2-8c85-280518c4aac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.644870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-863c13c9-a9e2-4fb2-8c85-280518c4aac8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:14.646190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-863c13c9-a9e2-4fb2-8c85-280518c4aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:14.716564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.719234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.726790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 428/866] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:14.745870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c13c9-a9e2-4fb2-8c85-280518c4aac8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.814222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.835216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:14.859101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 439/867] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:14.863465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:14.876413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c13c9-a9e2-4fb2-8c85-280518c4aac8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:14.928342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:14.929163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:14.929557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:14.930000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:14.930756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:14.932590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:14.976881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11982bd-185c-476d-b9a8-26cbfb77a4cc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.002916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 429/868] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:14 2022] POST /identity/v3/services => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:15:15.036858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.037969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.038561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.039051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.052340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:15.054319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:15.072590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:15.073590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:15.074530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:15.074530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:15.074530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:15.076865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 440/869] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:15.083022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.198066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816b82d-9d0e-4ffe-befb-6386c4d49ed7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.209832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:15.234250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.234896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.235198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.235510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.236156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:15.236458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:15.237830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:15.251320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 430/870] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:15.256824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.294297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:15.366745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea38b7-7cde-4fd6-8832-367e34ac133b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.404008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.404630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.404905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.405173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.412591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:15.414026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:15.425388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:15.426316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:15.427245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 441/871] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:15.433861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.539242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4712fd8-895a-4a61-8b4f-c84bd151fd99 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.575142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520f65e-5917-41cb-95d7-2d28c9c67a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.576108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520f65e-5917-41cb-95d7-2d28c9c67a4f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.576591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520f65e-5917-41cb-95d7-2d28c9c67a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.577071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520f65e-5917-41cb-95d7-2d28c9c67a4f None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.578583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520f65e-5917-41cb-95d7-2d28c9c67a4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:15.579273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520f65e-5917-41cb-95d7-2d28c9c67a4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:15.580909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520f65e-5917-41cb-95d7-2d28c9c67a4f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:15.593595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 431/872] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:15.600504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.703849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:15.710836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d9ef-4a97-404c-b576-6f9f1013157c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:15.747782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6629feb-08dc-4ab7-821d-0a6b8115c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:15.748383 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629feb-08dc-4ab7-821d-0a6b8115c573 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.748659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629feb-08dc-4ab7-821d-0a6b8115c573 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.748928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629feb-08dc-4ab7-821d-0a6b8115c573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.749654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6629feb-08dc-4ab7-821d-0a6b8115c573 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=6aa66d4b5bad476183509a4e00c56967) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:15.750613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6629feb-08dc-4ab7-821d-0a6b8115c573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:15.752031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6629feb-08dc-4ab7-821d-0a6b8115c573 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:15.760390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 442/873] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/role_assignments?user.id=6aa66d4b5bad476183509a4e00c56967&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:15.799659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025a05e-d31b-452f-816d-248855f4806d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.800276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025a05e-d31b-452f-816d-248855f4806d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.800276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025a05e-d31b-452f-816d-248855f4806d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.801863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 432/874] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:15.807485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f876d9-50f4-417d-9cd0-56bdde439513 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:15.807791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f876d9-50f4-417d-9cd0-56bdde439513 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:15.808046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f876d9-50f4-417d-9cd0-56bdde439513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:15.830389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-62f876d9-50f4-417d-9cd0-56bdde439513 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:15.831683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62f876d9-50f4-417d-9cd0-56bdde439513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:15.910637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 443/875] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:15.933399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025a05e-d31b-452f-816d-248855f4806d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:15.967655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:16.044142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025a05e-d31b-452f-816d-248855f4806d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:16.079248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:16.080112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:16.080580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:16.081000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.081722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:16.083257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:16.092395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:16.094114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:16.095154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:16.097536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 433/876] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:16.106419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f876d9-50f4-417d-9cd0-56bdde439513 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:16.198468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:16.212797 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f876d9-50f4-417d-9cd0-56bdde439513 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:16.250583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed304e-4b8c-4fc3-b323-553e157c6213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:16.251501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed304e-4b8c-4fc3-b323-553e157c6213 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:16.251790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed304e-4b8c-4fc3-b323-553e157c6213 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:16.252063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed304e-4b8c-4fc3-b323-553e157c6213 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.252700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed304e-4b8c-4fc3-b323-553e157c6213 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:16.253002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed304e-4b8c-4fc3-b323-553e157c6213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:16.254378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed304e-4b8c-4fc3-b323-553e157c6213 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:16.259760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 444/877] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:16.265498 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:16.368145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:16.375146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b69e5-ee48-4d53-b3b8-40f32ad7503e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:16.411979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb27dc-5eb9-41dd-a1a5-11f7eb2c7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:16.412584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb27dc-5eb9-41dd-a1a5-11f7eb2c7b23 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:16.412866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb27dc-5eb9-41dd-a1a5-11f7eb2c7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:16.413140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb27dc-5eb9-41dd-a1a5-11f7eb2c7b23 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.413773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb27dc-5eb9-41dd-a1a5-11f7eb2c7b23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:16.414078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb27dc-5eb9-41dd-a1a5-11f7eb2c7b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:16.415484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb27dc-5eb9-41dd-a1a5-11f7eb2c7b23 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:16.420434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 434/878] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:16.746618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb3e43-474a-4b29-81c1-833c975fc93e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:16.747023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb3e43-474a-4b29-81c1-833c975fc93e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:16.747199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb3e43-474a-4b29-81c1-833c975fc93e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.748770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 445/879] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:16.754711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03217f-8f54-468b-aa16-c1530e31d45a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:16.755001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03217f-8f54-468b-aa16-c1530e31d45a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:16.755276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03217f-8f54-468b-aa16-c1530e31d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:16.774123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cc03217f-8f54-468b-aa16-c1530e31d45a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:16.775348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc03217f-8f54-468b-aa16-c1530e31d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:16.846549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 435/880] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:16.868760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb3e43-474a-4b29-81c1-833c975fc93e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:16.981818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb3e43-474a-4b29-81c1-833c975fc93e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:17.020659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3228c7-a365-448f-8db8-3471249221f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:17.021182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3228c7-a365-448f-8db8-3471249221f7 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.021463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3228c7-a365-448f-8db8-3471249221f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.021737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3228c7-a365-448f-8db8-3471249221f7 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.022421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3228c7-a365-448f-8db8-3471249221f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:17.023857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3228c7-a365-448f-8db8-3471249221f7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:17.115042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 446/881] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:16 2022] POST /identity/v3/services => generated 269 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:17.170829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14e5c-7493-4d3f-8065-66b86ee53f71 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.171188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14e5c-7493-4d3f-8065-66b86ee53f71 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.171386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea14e5c-7493-4d3f-8065-66b86ee53f71 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.173663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 436/882] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:17.179489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa7bb0-72f7-47ca-ab46-dc21ccad5971 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.179771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa7bb0-72f7-47ca-ab46-dc21ccad5971 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.180048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa7bb0-72f7-47ca-ab46-dc21ccad5971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.200476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-72fa7bb0-72f7-47ca-ab46-dc21ccad5971 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:17.201701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-72fa7bb0-72f7-47ca-ab46-dc21ccad5971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:17.211342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:17.289295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 447/883] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:17.311730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14e5c-7493-4d3f-8065-66b86ee53f71 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:17.420455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:17.433934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea14e5c-7493-4d3f-8065-66b86ee53f71 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:17.471057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:17.472060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.472536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.473037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.473816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:17.475409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:17.485055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:17.486052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:17.486907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:17.489957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 437/884] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:17.495851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa7bb0-72f7-47ca-ab46-dc21ccad5971 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:17.601741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa7bb0-72f7-47ca-ab46-dc21ccad5971 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:17.637345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:17.637951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.638229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.638605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.639237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:17.639540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:17.641804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:17.648437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 448/885] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:17.654157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:17.764506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926ada6-f8ae-4942-8e71-f93996de7ee3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:17.803197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f204053-b07d-4cd3-9795-c966c2f3f23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:17.804170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f204053-b07d-4cd3-9795-c966c2f3f23e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.804788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f204053-b07d-4cd3-9795-c966c2f3f23e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.805269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f204053-b07d-4cd3-9795-c966c2f3f23e None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.806091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f204053-b07d-4cd3-9795-c966c2f3f23e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:17.806620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f204053-b07d-4cd3-9795-c966c2f3f23e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:17.808201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f204053-b07d-4cd3-9795-c966c2f3f23e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:17.813874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 438/886] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:17.819997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:17.838743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:17.884207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b7fbb-b047-490f-b47d-08c1d310e660 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.885151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b7fbb-b047-490f-b47d-08c1d310e660 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.885629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b7fbb-b047-490f-b47d-08c1d310e660 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.887437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 439/887] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:17.896271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39458a2-6e0a-49a0-b6b9-a896cf76ee5d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.896844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39458a2-6e0a-49a0-b6b9-a896cf76ee5d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.897320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39458a2-6e0a-49a0-b6b9-a896cf76ee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.926952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d39458a2-6e0a-49a0-b6b9-a896cf76ee5d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:17.927865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e56126-e8a9-4015-a820-5f1292f9f385 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:17.928626 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d39458a2-6e0a-49a0-b6b9-a896cf76ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:17.940672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:17.963343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:17.963938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:17.964379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:17.964778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:17.965676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] RBAC: Adding query filter params (interface=public, service_id=3bd6b6b6465b41a2a0a43f0de010eaca, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:17.966140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:17.967662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:17.974290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 449/888] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:15:17 2022] GET /identity/v3/endpoints?service_id=3bd6b6b6465b41a2a0a43f0de010eaca&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:17.981168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:18.016761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 440/889] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:18.040149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39458a2-6e0a-49a0-b6b9-a896cf76ee5d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:18.073908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:18.122110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03b9ac-0357-421d-99b7-842bfc3cd58a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:18.161116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:18.161303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39458a2-6e0a-49a0-b6b9-a896cf76ee5d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:18.162189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.162779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.163351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.164155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:18.165724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:18.177617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 450/890] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:17 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:15:18.198868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:18.199503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.199779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.200045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.212824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:18.217560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:18.218601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:18.220011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 441/891] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 02:15:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:18.227926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:18.347989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e3151-201c-4312-b11a-9fc9e4a35622 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:18.381343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:18.388148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68bee5-1a3b-414c-b423-7ea1739502ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:18.388787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68bee5-1a3b-414c-b423-7ea1739502ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.389039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68bee5-1a3b-414c-b423-7ea1739502ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.389332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68bee5-1a3b-414c-b423-7ea1739502ba None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.389984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68bee5-1a3b-414c-b423-7ea1739502ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:18.390310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68bee5-1a3b-414c-b423-7ea1739502ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:18.392449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68bee5-1a3b-414c-b423-7ea1739502ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:18.404568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 451/892] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 02:15:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:18.410705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:18.420460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:18.515048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30a22e-bb55-467e-a447-a70b9ea2d685 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:18.517406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289faa9-ae79-43f0-9b72-8c98ad6ab63a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.517696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289faa9-ae79-43f0-9b72-8c98ad6ab63a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.517968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289faa9-ae79-43f0-9b72-8c98ad6ab63a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.519586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 452/893] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:18.524597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791eadf-d0c7-4157-8481-09f176944e5b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.524881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791eadf-d0c7-4157-8481-09f176944e5b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.525154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791eadf-d0c7-4157-8481-09f176944e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.544163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a791eadf-d0c7-4157-8481-09f176944e5b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:18.545343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a791eadf-d0c7-4157-8481-09f176944e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:18.558542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:18.559223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.559508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.559783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.571347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:18.573075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:18.590317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:18.591330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:18.592206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:18.592206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:18.592206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:18.597127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 442/894] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:18.597994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:18.634014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 453/895] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:18.657122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791eadf-d0c7-4157-8481-09f176944e5b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:18.671324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:18.730401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f648b07-f689-47f5-8017-b3210582f4ae None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:18.763793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791eadf-d0c7-4157-8481-09f176944e5b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:18.763994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:18.764534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.764829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.765094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.766797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:18.767104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:18.769454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:18.785955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 443/896] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:18.793681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:18.795319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:18.808301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:18.809288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.809766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.810256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.810973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:18.812608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:18.898515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 454/897] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:18 2022] POST /identity/v3/services => generated 241 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:15:18.925600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a3ee1-26f9-4379-8e63-b3d7629c03e7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:18.950793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:18.979675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:18.980320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:18.980655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:18.980931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:18.990070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:18.991727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:19.006733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:19.007743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:19.008703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 444/898] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:15:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:19.016399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.131916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71d02b-5a4b-40e1-8a03-c41ffcec9269 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:19.168071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96c426-dc97-4b34-ad4e-caa4f97c58af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:19.168672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96c426-dc97-4b34-ad4e-caa4f97c58af None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.168946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96c426-dc97-4b34-ad4e-caa4f97c58af None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.169213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96c426-dc97-4b34-ad4e-caa4f97c58af None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.169939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96c426-dc97-4b34-ad4e-caa4f97c58af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:19.170239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96c426-dc97-4b34-ad4e-caa4f97c58af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:19.171736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96c426-dc97-4b34-ad4e-caa4f97c58af None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.191282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 455/899] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:19.197208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.303090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca7a2d-d8b6-445a-a8ef-6872c5ca53be None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:19.315935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a43cd-a93f-4a6a-a5dc-8956556873f1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.316210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a43cd-a93f-4a6a-a5dc-8956556873f1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.316490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a43cd-a93f-4a6a-a5dc-8956556873f1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.318068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 456/900] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:19.322946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e352-0531-47bd-b471-a498413460c5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.323237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e352-0531-47bd-b471-a498413460c5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.323509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e352-0531-47bd-b471-a498413460c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.341262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:19.341866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.342143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.342436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.343057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bff7e352-0531-47bd-b471-a498413460c5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:19.344264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bff7e352-0531-47bd-b471-a498413460c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:19.384726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:19.401140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:19.430773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 457/901] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:19.438527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:19.440267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.448058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7e352-0531-47bd-b471-a498413460c5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.559467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7e352-0531-47bd-b471-a498413460c5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:19.594573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 445/902] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:19.596638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.602132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:19.603098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.603582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.604582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.605386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:19.607451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:19.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:19.621210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:19.622095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:19.628798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 458/903] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:19.629757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.666735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:19.688881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:19.716162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432fb02-c386-4e6b-b480-bc2048ba55d3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:19.758690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad24a28-0d1a-4ae1-a1f9-1629b136fff1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:19.762241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:19.763153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.763635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.764130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.775851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:19.777583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:19.793364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:19.794337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:19.795255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:19.795255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:19.795255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:19.801577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 446/904] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:19.802435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:19.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.803324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.803529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.803629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.804312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:19.804619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:19.805977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.816900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 459/905] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:19.817804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.910671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84139e-ea08-44fe-aa22-90039aa326ab None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:19.919930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:19.926400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01216fe-e426-493c-b0a6-8af2c8ca6ba7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:19.947083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:19.956678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:19.957288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.957571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.957920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.958691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:19.958899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:19.960341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.965562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:19.966531 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:19.967041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:19.967507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:19.968577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:19.969078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:19.970685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:19.980988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 447/906] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:19.980988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:19.981579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 460/907] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:19.982733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:20.086104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb9a0-9a81-4732-bf88-ef69902578d0 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.098303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7ed14-5a35-485d-a984-fa4a3fe2798e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.122898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.123545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.124315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.124315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.132254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.133451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.141534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.142982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.143439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.143908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.144781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] RBAC: Adding query filter params (interface=public, service_id=f521a850a90b41a0a36c291bfae3778d, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:20.145262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:20.146559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:20.147471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:20.148338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:20.149222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.152731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 448/908] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:20.153548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:20.160780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 461/909] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:15:19 2022] GET /identity/v3/endpoints?service_id=f521a850a90b41a0a36c291bfae3778d&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:20.161901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:20.192346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:20.202549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:20.258625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f8a1b-f6db-4112-9bbd-1b30dddd1d3a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.276230 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ab5fc-b47c-4697-a37a-03236b0060c5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.294926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.295532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.295811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.296080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.296825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:20.297135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.298593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.314427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 449/910] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:20.317540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:20.321396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409f462-b61d-4d8c-81d0-0d9a3919603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.323227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f462-b61d-4d8c-81d0-0d9a3919603e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.323227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f462-b61d-4d8c-81d0-0d9a3919603e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.323227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f462-b61d-4d8c-81d0-0d9a3919603e None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.323227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409f462-b61d-4d8c-81d0-0d9a3919603e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.324627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409f462-b61d-4d8c-81d0-0d9a3919603e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.338491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 462/911] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:20 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:15:20.358840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa259-c149-46b6-bf60-ceeecc6f60e7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.359773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa259-c149-46b6-bf60-ceeecc6f60e7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.359920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa259-c149-46b6-bf60-ceeecc6f60e7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.365823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 463/912] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:20.367269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6474168-a7cf-4af3-98e1-3a11601900e1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.367552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6474168-a7cf-4af3-98e1-3a11601900e1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.367824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6474168-a7cf-4af3-98e1-3a11601900e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.391508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b6474168-a7cf-4af3-98e1-3a11601900e1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:20.393451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b6474168-a7cf-4af3-98e1-3a11601900e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:20.447216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c303c5-6ef9-4404-8e9b-b7c98f95f4b1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.458668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:20.480194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 464/913] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:20.497285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.498240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.498872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.499359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.500712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] RBAC: Adding query filter params (role.id=8611063528db47a3aba26d87c4f05a9f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=6aa66d4b5bad476183509a4e00c56967) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:20.501248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.502993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.504596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6474168-a7cf-4af3-98e1-3a11601900e1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:20.510130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 450/914] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:20 2022] GET /identity/v3/role_assignments?user.id=6aa66d4b5bad476183509a4e00c56967&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=8611063528db47a3aba26d87c4f05a9f => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:20.530752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:20.618752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6474168-a7cf-4af3-98e1-3a11601900e1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.657945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.658601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.658876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.659139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.659726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.661081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:20.670973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:20.672523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:20.673512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 465/915] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:20.678994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:20.786530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea341-8460-4591-99f6-7613386f5841 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.822279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.822902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.823174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.823493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.824119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:20.824415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.825777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.831812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 451/916] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:20.837397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:20.934736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:20.944789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f834fe1-5e1d-4044-89d2-ab5a1485e1db None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:20.979453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89cbd4-eb88-460e-9560-6f257aa76e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:20.980068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89cbd4-eb88-460e-9560-6f257aa76e6e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:20.980353 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89cbd4-eb88-460e-9560-6f257aa76e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:20.980634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89cbd4-eb88-460e-9560-6f257aa76e6e None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:20.981275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89cbd4-eb88-460e-9560-6f257aa76e6e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:20.981581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89cbd4-eb88-460e-9560-6f257aa76e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:20.982992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89cbd4-eb88-460e-9560-6f257aa76e6e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:20.989966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 466/917] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 02:15:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:20.995527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:21.034183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:21.077095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fbfcd-3b99-4fa8-82bc-9a92e3de8728 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:21.077300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fbfcd-3b99-4fa8-82bc-9a92e3de8728 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:21.077476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fbfcd-3b99-4fa8-82bc-9a92e3de8728 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:21.079175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 467/918] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:21.084171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9569d8-1d86-4bc8-97f3-433eddaa452b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:21.084387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9569d8-1d86-4bc8-97f3-433eddaa452b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:21.084653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9569d8-1d86-4bc8-97f3-433eddaa452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:21.103580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3ad61-12c8-42a5-8a0f-b50f9d4993d1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:21.104226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ac9569d8-1d86-4bc8-97f3-433eddaa452b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:21.105418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ac9569d8-1d86-4bc8-97f3-433eddaa452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:21.141187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:21.142202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:21.142725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:21.143000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:21.143659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:21.145022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:21.189401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 468/919] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:21.212357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9569d8-1d86-4bc8-97f3-433eddaa452b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:21.224421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 452/920] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:21.309458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:21.334877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9569d8-1d86-4bc8-97f3-433eddaa452b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:21.380258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:21.380884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:21.381159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:21.381423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:21.381995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:21.383448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:21.393575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:21.394641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:21.395475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:21.398486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 469/921] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:21.401703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:21.506318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:21.517408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9369f0d-5692-453b-a340-6f519686f496 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:21.554065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:21.554719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:21.555067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:21.555319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:21.556080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:21.556402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:21.557849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:21.569215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 453/922] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:21.571326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:21.679475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da9af6-f683-4903-900c-02a49bf4c748 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:21.717528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:21.718125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:21.719112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:21.719594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:21.720334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:21.720637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:21.721994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:21.731727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 470/923] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:21.733062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:21.840953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c796815-c006-4cf3-ba11-897821f12028 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:21.875177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578e356-dc51-47ca-aaea-59a146565cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:21.875785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578e356-dc51-47ca-aaea-59a146565cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:21.876063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578e356-dc51-47ca-aaea-59a146565cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:21.876332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578e356-dc51-47ca-aaea-59a146565cf1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:21.877028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578e356-dc51-47ca-aaea-59a146565cf1 None admin] RBAC: Adding query filter params (interface=public, service_id=055e8fa5e0c0401db86557d8e2d65ab5, region_id=RegionOne) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:21.877333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578e356-dc51-47ca-aaea-59a146565cf1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:21.878743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578e356-dc51-47ca-aaea-59a146565cf1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:21.884021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 454/924] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:15:21 2022] GET /identity/v3/endpoints?service_id=055e8fa5e0c0401db86557d8e2d65ab5&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:21.889386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:21.924830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:22.013046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2e3a6-e7db-4888-b47e-b71e2f94e53d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:22.049900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1951ae-244c-4aa4-9b92-2366b0dfd8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:22.050989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1951ae-244c-4aa4-9b92-2366b0dfd8ea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:22.051514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1951ae-244c-4aa4-9b92-2366b0dfd8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:22.051976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1951ae-244c-4aa4-9b92-2366b0dfd8ea None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:22.052764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1951ae-244c-4aa4-9b92-2366b0dfd8ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:22.054300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1951ae-244c-4aa4-9b92-2366b0dfd8ea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:22.060891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 471/925] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:21 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:15:22.533581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1b48-ce39-4f0a-84c6-93e4ff8c9607 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:22.534574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1b48-ce39-4f0a-84c6-93e4ff8c9607 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:22.535055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1b48-ce39-4f0a-84c6-93e4ff8c9607 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:22.536895 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 455/926] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:22.542448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4c232-d0de-48d6-9810-7e27ad3e6148 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:22.543225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4c232-d0de-48d6-9810-7e27ad3e6148 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:22.543713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4c232-d0de-48d6-9810-7e27ad3e6148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:22.563445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0bc4c232-d0de-48d6-9810-7e27ad3e6148 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:22.564958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0bc4c232-d0de-48d6-9810-7e27ad3e6148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:22.610660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8728d3-5a34-4590-b5cb-3b12198aecc6 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:22.610987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8728d3-5a34-4590-b5cb-3b12198aecc6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:22.611217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8728d3-5a34-4590-b5cb-3b12198aecc6 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:22.612674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 456/927] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:22.618270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae55fe-afe0-4abd-b543-909cf81e6481 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:22.618591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae55fe-afe0-4abd-b543-909cf81e6481 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:22.618864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae55fe-afe0-4abd-b543-909cf81e6481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:22.630767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:22.639565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 472/928] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:22.644223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-92ae55fe-afe0-4abd-b543-909cf81e6481 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:22.644223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-92ae55fe-afe0-4abd-b543-909cf81e6481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:22.656645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:22.666453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4c232-d0de-48d6-9810-7e27ad3e6148 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:22.711914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 457/929] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:22.734453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae55fe-afe0-4abd-b543-909cf81e6481 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:22.779505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4c232-d0de-48d6-9810-7e27ad3e6148 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:22.820413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:22.821381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:22.821974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:22.822546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:22.823364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:22.824934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:22.835084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:22.836048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:15:22.836972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:22.845249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 473/930] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 8 02:15:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:22.845376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae55fe-afe0-4abd-b543-909cf81e6481 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:22.847412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:22.885547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:22.887287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:22.887766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:22.888427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:22.902311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:22.909575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:22.910536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:22.911405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:22.912038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 458/931] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 02:15:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:22.928484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:22.929755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:22.942046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:22.974852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8f830-d019-43ba-8339-c27792c57424 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.012987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.014067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.014339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.014774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.016247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:23.016639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.018187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.029984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 474/932] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:23.031031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.045362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5830248-8c77-4916-aaa8-5c6fee9a0152 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.083261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.084212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.084686 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.085142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.085999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:23.086517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.088072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.097224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 459/933] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 02:15:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:23.102380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.146762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45c479-3593-411c-b710-1fcffcf5583e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.173636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:23.176288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:23.186971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.188680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.189105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.189548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.190371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:23.190862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.192409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.203209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 475/934] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:23.204491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.210345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18398457-9d6e-4efb-971c-49241fd668da None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.248753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.249398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.250201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.250487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.264373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.266759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:23.284881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:23.286026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:23.287187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:23.287187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:23.287187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.290412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 460/935] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:23.293992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.331507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98706758-ac33-4546-8eb7-cad1df2fd4bd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.367175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.367861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.368335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.368590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.369357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.370862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.407448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c060be0-5fe5-4cc9-a93c-54ae9adac329 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.431387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:23.443893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910dd81-a011-471b-ac77-88738115271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.444604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dd81-a011-471b-ac77-88738115271a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.444994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dd81-a011-471b-ac77-88738115271a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.445258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dd81-a011-471b-ac77-88738115271a None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.445837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910dd81-a011-471b-ac77-88738115271a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:23.446133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910dd81-a011-471b-ac77-88738115271a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.450396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910dd81-a011-471b-ac77-88738115271a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.450747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 476/936] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:23.465044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 461/937] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:23.471895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.525594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:23.527343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793276-4740-4581-9578-4264347f773b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.527627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793276-4740-4581-9578-4264347f773b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.527894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793276-4740-4581-9578-4264347f773b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.530595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 462/938] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:23.536035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f13270-ed35-411b-99af-e1d98482d772 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.536662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f13270-ed35-411b-99af-e1d98482d772 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.536662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f13270-ed35-411b-99af-e1d98482d772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.559854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-73f13270-ed35-411b-99af-e1d98482d772 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:23.561757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-73f13270-ed35-411b-99af-e1d98482d772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:23.598738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76fc67-4527-4d76-9057-2f1354dc0083 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.643324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.644128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.644406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.645102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.654811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.658454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.660924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 463/939] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:23.677879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:23.678952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:23.679890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 477/940] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:15:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:23.684604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f13270-ed35-411b-99af-e1d98482d772 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.685160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.799042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f13270-ed35-411b-99af-e1d98482d772 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.813169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:23.814721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:23.818456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742bd59b-8b16-4679-b06b-31d9323991f3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:23.837777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.838759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.839202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.841558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.841558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.842405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.845347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING keystone.server.flask.common [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 02:15:23.859368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:23.860221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:23.860634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:23.860907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:23.861643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:23.862117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:23.864565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:23.887818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 478/941] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 02:15:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:23.890529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:23.894964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.sql.core [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:15:23.894964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 02:15:23.894964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [parameters: {'id': '09a18ffcda9c4338b703d62e1045f1e3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 02:15:23.894964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 09a18ffcda9c4338b703d62e1045f1e3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 09a18ffcda9c4338b703d62e1045f1e3. Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:15:23.903818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 02:15:23.905344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application yield resource Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise self.value Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application session.commit() Aug 08 02:15:23.906338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:15:23.907264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:15:23.908152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:15:23.910744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [parameters: {'id': '09a18ffcda9c4338b703d62e1045f1e3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:23.913051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:23.915110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 09a18ffcda9c4338b703d62e1045f1e3. Aug 08 02:15:23.917546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:23.920229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 464/942] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Aug 8 02:15:23 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:23.929865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.021662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fb30f-5f87-459c-b640-e2775698bf81 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.061658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bce3b-0b39-4c69-ae6f-24a51148abea None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.069393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38f618-098c-48f0-a294-6f001fed3318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.070113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38f618-098c-48f0-a294-6f001fed3318 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.070473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38f618-098c-48f0-a294-6f001fed3318 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.070833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38f618-098c-48f0-a294-6f001fed3318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.085264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38f618-098c-48f0-a294-6f001fed3318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.087228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38f618-098c-48f0-a294-6f001fed3318 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.102266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190870c-1958-4193-a1e3-37de5a8a8a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.103264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190870c-1958-4193-a1e3-37de5a8a8a03 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.103795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190870c-1958-4193-a1e3-37de5a8a8a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.104239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190870c-1958-4193-a1e3-37de5a8a8a03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ad38f618-098c-48f0-a294-6f001fed3318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:24.106105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:24.107278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:24.108304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:24.108304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:24.108304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:24.108484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 479/943] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:24.109987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d45e-2af6-446f-80b2-feb1113c7de0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.110268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d45e-2af6-446f-80b2-feb1113c7de0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.110574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d45e-2af6-446f-80b2-feb1113c7de0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.111930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 480/944] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:24.116828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d45e-2af6-446f-80b2-feb1113c7de0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.119153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190870c-1958-4193-a1e3-37de5a8a8a03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.120822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190870c-1958-4193-a1e3-37de5a8a8a03 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.130476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4190870c-1958-4193-a1e3-37de5a8a8a03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:24.132398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:24.133467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:24.134600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:24.138483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 465/945] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:24.144013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476928dc-b484-49e5-a436-c6d7330d04b3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.144013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476928dc-b484-49e5-a436-c6d7330d04b3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.144013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476928dc-b484-49e5-a436-c6d7330d04b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.160230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-476928dc-b484-49e5-a436-c6d7330d04b3 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:24.161793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-476928dc-b484-49e5-a436-c6d7330d04b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:24.186169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:24.229403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d45e-2af6-446f-80b2-feb1113c7de0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.238591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 466/946] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:24.238591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476928dc-b484-49e5-a436-c6d7330d04b3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.276335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.276931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.277205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.277472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.278104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:24.278433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.279791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.292312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 481/947] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:24.292312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.348263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476928dc-b484-49e5-a436-c6d7330d04b3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.370591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:24.391011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.391994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.392274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.392546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.393370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:24.393676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.395794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.402703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d534b7-26c1-4fd3-8f53-be25c7744114 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.430467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 467/948] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:24.430467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.441453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.442191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.442572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.443048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.443692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.445157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.450628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:24.457375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:24.458665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:24.459821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 482/949] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:24.464588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.558557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f5a6-91a4-483d-a51b-3696d780b64b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.597303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6e19f-c5d2-4d78-a5be-a66ec5d228a7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.603102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.604062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.604575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.605096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.614444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.616243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:24.629542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:24.630712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:24.631928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:24.632700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 468/950] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:24.639142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.647892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.648578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.648923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.649261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.649957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:24.650910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.652278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.653638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:24.659787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 483/951] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:24.665405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.756666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:24.786608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fe9bd-d5aa-4166-b0cb-adc701f60ddc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.787391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe7cd4-91c7-4bca-9b69-7eaefda6b82f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:24.828057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.828763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.829115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.829456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.830210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:24.830607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.832478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.834156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:24.835001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:24.835724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:24.836060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:24.836687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:24.837029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:24.838386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:24.845938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 484/952] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:24.851612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.853311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 469/953] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:24.859784 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:24.940377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:24.991301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd61bc-68ba-4ec6-b290-64958bfbf0e8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:25.001628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf10c-5c6f-4916-9b5a-d408e69b6e47 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:25.027977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921fc01-d6fa-453c-99bd-240da6f96961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:25.028499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921fc01-d6fa-453c-99bd-240da6f96961 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:25.028778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921fc01-d6fa-453c-99bd-240da6f96961 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:25.029049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921fc01-d6fa-453c-99bd-240da6f96961 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:25.029707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921fc01-d6fa-453c-99bd-240da6f96961 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:25.031258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921fc01-d6fa-453c-99bd-240da6f96961 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:25.066751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd365b20-b2ba-422a-9b5a-b8cf63ffe973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:25.067794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd365b20-b2ba-422a-9b5a-b8cf63ffe973 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:25.068342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd365b20-b2ba-422a-9b5a-b8cf63ffe973 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:25.068803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd365b20-b2ba-422a-9b5a-b8cf63ffe973 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/6aa66d4b5bad476183509a4e00c56967/roles/8611063528db47a3aba26d87c4f05a9f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:25.074411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd365b20-b2ba-422a-9b5a-b8cf63ffe973 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=6aa66d4b5bad476183509a4e00c56967, role_id=8611063528db47a3aba26d87c4f05a9f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:25.074719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd365b20-b2ba-422a-9b5a-b8cf63ffe973 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:25.102252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 485/954] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:25.147252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:25.153101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 470/955] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:24 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/6aa66d4b5bad476183509a4e00c56967/roles/8611063528db47a3aba26d87c4f05a9f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:15:25.616900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360243bb-0332-4e3e-897a-ad2580f3a9c9 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:25.617308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360243bb-0332-4e3e-897a-ad2580f3a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:25.617488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360243bb-0332-4e3e-897a-ad2580f3a9c9 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:25.619118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 486/956] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:25.624384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52dac0-3bf1-492a-b99b-94b0cca7f589 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:25.624867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52dac0-3bf1-492a-b99b-94b0cca7f589 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:25.625065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52dac0-3bf1-492a-b99b-94b0cca7f589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:25.643824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-dd52dac0-3bf1-492a-b99b-94b0cca7f589 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:25.645015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dd52dac0-3bf1-492a-b99b-94b0cca7f589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:25.720332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 471/957] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:25.742722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360243bb-0332-4e3e-897a-ad2580f3a9c9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:25.854877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360243bb-0332-4e3e-897a-ad2580f3a9c9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:25.872928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:25.891106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:25.891728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:25.892006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:25.892277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:25.892859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:25.894250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:25.902990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:25.905222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:25.906644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 487/958] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:25.910226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52dac0-3bf1-492a-b99b-94b0cca7f589 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:26.022301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52dac0-3bf1-492a-b99b-94b0cca7f589 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:26.047864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:26.059426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97866b1b-f1e4-481b-913a-6c64f3294486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:26.060135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866b1b-f1e4-481b-913a-6c64f3294486 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:26.060413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866b1b-f1e4-481b-913a-6c64f3294486 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:26.060680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97866b1b-f1e4-481b-913a-6c64f3294486 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:26.061301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97866b1b-f1e4-481b-913a-6c64f3294486 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:26.061598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97866b1b-f1e4-481b-913a-6c64f3294486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:26.063204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97866b1b-f1e4-481b-913a-6c64f3294486 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:26.068411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 472/959] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:26.074090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:26.177308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4412e71-0064-4158-8510-2ba6b9942d24 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:26.210859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954ed08-2b66-4065-a487-bf9f50847814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:26.211455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954ed08-2b66-4065-a487-bf9f50847814 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:26.211764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954ed08-2b66-4065-a487-bf9f50847814 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:26.212034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954ed08-2b66-4065-a487-bf9f50847814 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:26.212657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954ed08-2b66-4065-a487-bf9f50847814 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:26.212953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954ed08-2b66-4065-a487-bf9f50847814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:26.214292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954ed08-2b66-4065-a487-bf9f50847814 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:26.219602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 488/960] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:26.711433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9fb6-c2fc-40bb-9322-fb28ddd8718d None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:26.711972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9fb6-c2fc-40bb-9322-fb28ddd8718d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:26.711972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9fb6-c2fc-40bb-9322-fb28ddd8718d None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:26.713533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 473/961] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:26.719312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a42a-ef4d-415f-91e0-305d7936a309 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:26.719600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a42a-ef4d-415f-91e0-305d7936a309 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:26.719875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a42a-ef4d-415f-91e0-305d7936a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:26.738620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b851a42a-ef4d-415f-91e0-305d7936a309 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:26.740066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b851a42a-ef4d-415f-91e0-305d7936a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:26.784352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:26.809023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 489/962] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:26.834108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a9fb6-c2fc-40bb-9322-fb28ddd8718d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:26.945596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a9fb6-c2fc-40bb-9322-fb28ddd8718d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:26.982086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:26.982710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:26.982988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:26.983259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:26.984584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:26.986021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:27.026036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.sql.core [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:15:27.026036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 02:15:27.026036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [parameters: {'user_id': '65e599eb18b54700b6ce2903934391c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 02:15:27.026036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80267) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Aug 08 02:15:27.032884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:15:27.033738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application yield resource Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise self.value Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application session.commit() Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:15:27.034704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:15:27.035588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:15:27.036503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application result.read() Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:15:27.037370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [parameters: {'user_id': '65e599eb18b54700b6ce2903934391c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:27.038224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 02:15:27.040948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 02:15:27.043191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 02:15:27.043191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 02:15:27.043191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:15:27.043191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:15:27.043191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 02:15:27.043191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:27.043191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 474/963] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 02:15:26 2022] POST /identity/v3/users => generated 160 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:27.044058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a42a-ef4d-415f-91e0-305d7936a309 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.145452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a42a-ef4d-415f-91e0-305d7936a309 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:27.180485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ac499-7478-4efa-a2ac-8f1bb8ea1c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:27.181112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ac499-7478-4efa-a2ac-8f1bb8ea1c4d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.181389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ac499-7478-4efa-a2ac-8f1bb8ea1c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.181662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ac499-7478-4efa-a2ac-8f1bb8ea1c4d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.189671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ac499-7478-4efa-a2ac-8f1bb8ea1c4d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:27.191227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ac499-7478-4efa-a2ac-8f1bb8ea1c4d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-277ac499-7478-4efa-a2ac-8f1bb8ea1c4d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:27.203204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:27.204150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:27.205072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 490/964] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:27.210560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.267187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:27.316741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97390-7fc3-4068-baee-585d01212653 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:27.353313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740314d3-efcd-4aea-8fed-bf9c1b7b98f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:27.354297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740314d3-efcd-4aea-8fed-bf9c1b7b98f3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.354797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740314d3-efcd-4aea-8fed-bf9c1b7b98f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.355250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740314d3-efcd-4aea-8fed-bf9c1b7b98f3 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.356209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740314d3-efcd-4aea-8fed-bf9c1b7b98f3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:27.356712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740314d3-efcd-4aea-8fed-bf9c1b7b98f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:27.359135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740314d3-efcd-4aea-8fed-bf9c1b7b98f3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:27.369241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467585f-8262-477c-ad3c-f5aa4d13a3d1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.369581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467585f-8262-477c-ad3c-f5aa4d13a3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.369861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467585f-8262-477c-ad3c-f5aa4d13a3d1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.371583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 491/965] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:27.377966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd60cb-96a3-451f-9dd7-994d61d0c19e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.378629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd60cb-96a3-451f-9dd7-994d61d0c19e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.379099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd60cb-96a3-451f-9dd7-994d61d0c19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.399956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a4cd60cb-96a3-451f-9dd7-994d61d0c19e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:27.401530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a4cd60cb-96a3-451f-9dd7-994d61d0c19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:27.401978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 475/966] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:27.436245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:27.480735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e154b93-7154-4fa5-9107-6651d708b7ef None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.481025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e154b93-7154-4fa5-9107-6651d708b7ef None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.481305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e154b93-7154-4fa5-9107-6651d708b7ef None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.486827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 476/968] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:27.487050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 492/968] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:27.488507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a421-0b7b-46af-a235-8d1289ac63ea None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.489053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a421-0b7b-46af-a235-8d1289ac63ea None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.489329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a421-0b7b-46af-a235-8d1289ac63ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.508965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd60cb-96a3-451f-9dd7-994d61d0c19e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.512394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0544a421-0b7b-46af-a235-8d1289ac63ea None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:27.514231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0544a421-0b7b-46af-a235-8d1289ac63ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:27.598928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 477/969] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:27.625980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a421-0b7b-46af-a235-8d1289ac63ea None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.634480 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd60cb-96a3-451f-9dd7-994d61d0c19e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:27.679083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:27.679963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.680415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.680834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.681566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:27.683713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:27.684418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:27.695251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:27.696529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:27.697573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:27.713166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 493/970] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:27.713166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.743237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:27.754883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a421-0b7b-46af-a235-8d1289ac63ea None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:27.791400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:27.792027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.792563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.792563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.802976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:27.807644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:27.808581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:27.809421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 478/971] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:27.814758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.820218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da1450-3e6d-41df-8942-e86cc98ff239 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:27.857067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:27.857591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.858276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.858628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.859291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:27.859655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:27.861046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:27.867180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 494/972] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:27.872610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.926197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:27.930425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b2b3-b8c2-4518-bf4f-7cae1a48c2de None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:27.965539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:27.966168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:27.966483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:27.966757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:27.967417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:27.967717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:27.969647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:27.978236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 479/973] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:27.981149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:27.983619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:27.989532 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d19eaf-88d5-4fa7-bd7b-6cb2dac0e2f4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.030121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.031029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.031505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.031996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.032762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:28.033075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.034469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.039838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 495/974] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:28.045538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:28.093270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7688f9-9412-4380-ad9d-71293fe3d95e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.127331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.127885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.128165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.128437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.138763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.140876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.152602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5776a-2193-432e-968a-10612caaf6b8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:28.154712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:28.155750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:28.157078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:28.157078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:28.157078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:28.157078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 480/975] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:28.161630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:28.176203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:28.198305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.199307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.199762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.200198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.201078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] RBAC: Adding query filter params (interface=admin, service_id=055e8fa5e0c0401db86557d8e2d65ab5, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:28.201557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.204116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.212684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 496/976] 10.222.0.15 () {56 vars in 1249 bytes} [Mon Aug 8 02:15:28 2022] GET /identity/v3/endpoints?service_id=055e8fa5e0c0401db86557d8e2d65ab5&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:28.216968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:28.243273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:28.272225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bfeb1-ab17-40ff-82c1-c9130b223b63 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.308335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4211c-192d-40af-a3b3-49444caff2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.308937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4211c-192d-40af-a3b3-49444caff2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.309215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4211c-192d-40af-a3b3-49444caff2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.309485 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4211c-192d-40af-a3b3-49444caff2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.310120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4211c-192d-40af-a3b3-49444caff2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:28.310454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4211c-192d-40af-a3b3-49444caff2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.312377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4211c-192d-40af-a3b3-49444caff2c9 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.322178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9936dd-edcb-4126-ace2-08ce0e3b292b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.324426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 481/977] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:28.325640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51c66-5ec4-44d8-ba93-30928129cacc None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.325923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51c66-5ec4-44d8-ba93-30928129cacc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.326197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51c66-5ec4-44d8-ba93-30928129cacc None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.328170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 482/978] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:28.330295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51c66-5ec4-44d8-ba93-30928129cacc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:28.359640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25fac93-ba47-44fb-bd26-37a8d47f57af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.360464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fac93-ba47-44fb-bd26-37a8d47f57af None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.360740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fac93-ba47-44fb-bd26-37a8d47f57af None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.361010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fac93-ba47-44fb-bd26-37a8d47f57af None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.361608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25fac93-ba47-44fb-bd26-37a8d47f57af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.363026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25fac93-ba47-44fb-bd26-37a8d47f57af None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.369318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 497/979] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:28 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:15:28.370288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33134e44-0a80-4775-8ccd-09f6c46432c3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.370594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33134e44-0a80-4775-8ccd-09f6c46432c3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.370862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33134e44-0a80-4775-8ccd-09f6c46432c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.391213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-33134e44-0a80-4775-8ccd-09f6c46432c3 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:28.392423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-33134e44-0a80-4775-8ccd-09f6c46432c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:28.413106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:28.446976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51c66-5ec4-44d8-ba93-30928129cacc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.482382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 498/980] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:28.484449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.485093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.485371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.485642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.494492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33134e44-0a80-4775-8ccd-09f6c46432c3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:28.494682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.496010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:28.509295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:28.510341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:28.511329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 483/981] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:15:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:28.516342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:28.517650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:28.608827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33134e44-0a80-4775-8ccd-09f6c46432c3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.638191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83542a3b-06db-421a-a62e-20b47c8a25ca None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.645834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.646700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.646978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.647249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.647891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.649246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.669609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:28.674960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.675557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.675835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.676102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.676882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:28.677182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.678638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:28.696593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 484/982] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 02:15:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:28.701928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 499/983] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:28 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:15:28.703074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:28.815736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c207d4e-0e1a-4954-af82-32e68375f18e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:28.859216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:28.859826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:28.860132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:28.860408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:28.925416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:28.939279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:28.946826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:29.044663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 485/984] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:28 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:15:29.047100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:29.126050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:29.149946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad1985-d79b-4a75-8b51-c4de014bb107 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:29.184246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:29.184801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:29.185457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:29.186081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:29.198239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:29.199724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:29.214106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:29.215083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:29.216090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:29.216090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:29.216090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:29.218506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 500/985] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:29.221952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:29.305726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:29.320200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6fdc1-3229-4ddb-a1bf-6c104d28c553 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:29.355446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:29.356047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:29.356321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:29.356585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:29.357208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:29.357503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:29.359201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:29.375247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 486/986] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:29.376525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:29.480598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c6982-c141-43f7-90bb-51569a8bd02a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:29.510753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:29.519011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:29.519632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:29.519967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:29.520293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:29.526832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:29.528314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:29.539451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:29.540370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:29.541861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 501/987] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:29.550497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:29.651161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac1704-0f42-497d-b952-561fc11d43f1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:29.685415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e52e0-a3ef-4cc3-858e-fa475a2977a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:29.686133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e52e0-a3ef-4cc3-858e-fa475a2977a7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:29.686567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e52e0-a3ef-4cc3-858e-fa475a2977a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:29.686944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e52e0-a3ef-4cc3-858e-fa475a2977a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:29.687741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e52e0-a3ef-4cc3-858e-fa475a2977a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:29.688146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e52e0-a3ef-4cc3-858e-fa475a2977a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:29.689663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e52e0-a3ef-4cc3-858e-fa475a2977a7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:29.701754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 487/988] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:29.707721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:29.820450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d179b5-4602-4a65-ba0f-04436c163304 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:29.856077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65918f-4327-40ce-89dc-52a7e7893f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:29.856746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65918f-4327-40ce-89dc-52a7e7893f9a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:29.857029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65918f-4327-40ce-89dc-52a7e7893f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:29.857302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65918f-4327-40ce-89dc-52a7e7893f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:29.858042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65918f-4327-40ce-89dc-52a7e7893f9a None admin] RBAC: Adding query filter params (role.id=8611063528db47a3aba26d87c4f05a9f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=6aa66d4b5bad476183509a4e00c56967) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:29.858365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65918f-4327-40ce-89dc-52a7e7893f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:29.859873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65918f-4327-40ce-89dc-52a7e7893f9a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:29.866769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 502/989] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:29 2022] GET /identity/v3/role_assignments?user.id=6aa66d4b5bad476183509a4e00c56967&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=8611063528db47a3aba26d87c4f05a9f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:30.402770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714301bc-a8f9-46e5-b778-bfaad8ce40e7 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:30.403325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714301bc-a8f9-46e5-b778-bfaad8ce40e7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:30.403325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714301bc-a8f9-46e5-b778-bfaad8ce40e7 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:30.404784 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 488/990] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:30.409925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800abf0-a675-462c-bfd0-63ef16c85a68 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:30.410330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800abf0-a675-462c-bfd0-63ef16c85a68 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:30.410537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800abf0-a675-462c-bfd0-63ef16c85a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:30.428329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0800abf0-a675-462c-bfd0-63ef16c85a68 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:30.429496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0800abf0-a675-462c-bfd0-63ef16c85a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:30.479176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:30.498869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 503/991] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:30.521955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714301bc-a8f9-46e5-b778-bfaad8ce40e7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:30.530936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:30.627223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714301bc-a8f9-46e5-b778-bfaad8ce40e7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:30.660813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4170d61-70c6-4703-ae7a-cecadb4cd951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:30.661363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4170d61-70c6-4703-ae7a-cecadb4cd951 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:30.661642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4170d61-70c6-4703-ae7a-cecadb4cd951 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:30.661911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4170d61-70c6-4703-ae7a-cecadb4cd951 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:30.662513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4170d61-70c6-4703-ae7a-cecadb4cd951 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:30.663874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4170d61-70c6-4703-ae7a-cecadb4cd951 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-d4170d61-70c6-4703-ae7a-cecadb4cd951 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:30.673149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:30.674219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:30.675166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 489/992] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 02:15:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:30.680193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800abf0-a675-462c-bfd0-63ef16c85a68 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:30.759761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc86e1-efe3-4939-826c-2bc3ca2ab0c1 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:30.760103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc86e1-efe3-4939-826c-2bc3ca2ab0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:30.760312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc86e1-efe3-4939-826c-2bc3ca2ab0c1 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:30.761786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 490/993] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:30.766862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe21552-d98b-4f89-9101-8885dfb87aa6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:30.767146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe21552-d98b-4f89-9101-8885dfb87aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:30.767420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe21552-d98b-4f89-9101-8885dfb87aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:30.781911 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800abf0-a675-462c-bfd0-63ef16c85a68 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:30.785487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-abe21552-d98b-4f89-9101-8885dfb87aa6 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:30.786707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-abe21552-d98b-4f89-9101-8885dfb87aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:30.818890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:30.819492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:30.819810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:30.820076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:30.820647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:30.820952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:30.822492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:30.832645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 504/994] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:30.834603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:30.857205 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:30.862400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 491/995] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:30.884185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:30.885585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe21552-d98b-4f89-9101-8885dfb87aa6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:30.942484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab02321-3ac6-4ed9-a498-89c8722ccdda None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:30.980349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:30.980949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:30.981223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:30.981497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:30.982128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:30.982460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:30.983992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:30.993763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 505/996] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 02:15:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:30.994957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:31.001634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe21552-d98b-4f89-9101-8885dfb87aa6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:31.038169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:31.038819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.039096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.039363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.039930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:31.041284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:31.049921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:31.051184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:31.052028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 492/997] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:31.056892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:31.116277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da3076-c2f1-4bdd-9197-358a1189a274 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:31.126771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:31.128834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:31.153722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:31.154326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.154637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.154909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.156084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:31.157472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:31.164452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc6462-c2d2-4ca1-be0a-a7c047f3b9ad None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:31.198554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:31.199084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.199362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.199646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.200246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:31.200544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:31.201895 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:31.207308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 493/998] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:31.212917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:31.246379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 506/999] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:31.323155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f539dad-5cd9-4e2c-b463-77afbf4a76b7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:31.352500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:31.368878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:31.369490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.369766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.370035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.370696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:31.370997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:31.372383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:31.382435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 494/1000] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:31.385239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:31.497285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43982aed-60e6-4942-8441-b4693f620683 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:31.532445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18474dd-a7e5-4b8a-9b1e-ac1cc2771317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:31.533090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18474dd-a7e5-4b8a-9b1e-ac1cc2771317 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.533372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18474dd-a7e5-4b8a-9b1e-ac1cc2771317 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.533640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18474dd-a7e5-4b8a-9b1e-ac1cc2771317 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.534332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18474dd-a7e5-4b8a-9b1e-ac1cc2771317 None admin] RBAC: Adding query filter params (interface=public, service_id=51a504d815b044da853e7d60ef361cf0, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:31.534667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18474dd-a7e5-4b8a-9b1e-ac1cc2771317 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:31.536048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18474dd-a7e5-4b8a-9b1e-ac1cc2771317 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:31.545493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 507/1001] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:15:31 2022] GET /identity/v3/endpoints?service_id=51a504d815b044da853e7d60ef361cf0&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:31.547602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:31.648519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072fbd-c4df-437e-950a-1f1ab60d8985 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:31.681892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88309e42-6b51-455b-b76a-a382284bbbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:31.682528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88309e42-6b51-455b-b76a-a382284bbbfa None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.682813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88309e42-6b51-455b-b76a-a382284bbbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.683085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88309e42-6b51-455b-b76a-a382284bbbfa None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.683684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88309e42-6b51-455b-b76a-a382284bbbfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:31.685098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88309e42-6b51-455b-b76a-a382284bbbfa None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:31.698375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 495/1002] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:31 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:15:31.940431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b8ff-59bb-43c8-a423-dcc5cc7fb8fc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.940717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b8ff-59bb-43c8-a423-dcc5cc7fb8fc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.940974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b8ff-59bb-43c8-a423-dcc5cc7fb8fc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.942482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 508/1003] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:31.947415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2337266-2ff3-4b3c-93d6-549bdb9e7a53 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:31.947707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2337266-2ff3-4b3c-93d6-549bdb9e7a53 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:31.947983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2337266-2ff3-4b3c-93d6-549bdb9e7a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:31.967241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a2337266-2ff3-4b3c-93d6-549bdb9e7a53 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:31.968737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a2337266-2ff3-4b3c-93d6-549bdb9e7a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:32.017640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:32.037192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 496/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:32.058842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b8ff-59bb-43c8-a423-dcc5cc7fb8fc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:32.095983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:32.164650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b8ff-59bb-43c8-a423-dcc5cc7fb8fc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:32.199742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:32.200288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:32.200566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:32.200838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] PATH_INFO: `/v3/services/image` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:32.201416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:32.203172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:32.211826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:32.212823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:32.218757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 509/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 8 02:15:32 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:32.219551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2337266-2ff3-4b3c-93d6-549bdb9e7a53 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:32.318911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2337266-2ff3-4b3c-93d6-549bdb9e7a53 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:32.353752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83688297-dfe4-4910-8e53-a879c7eb3594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:32.354382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83688297-dfe4-4910-8e53-a879c7eb3594 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:32.354661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83688297-dfe4-4910-8e53-a879c7eb3594 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:32.354931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83688297-dfe4-4910-8e53-a879c7eb3594 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:32.355559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83688297-dfe4-4910-8e53-a879c7eb3594 None admin] RBAC: Adding query filter params (name=image) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:32.355861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83688297-dfe4-4910-8e53-a879c7eb3594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:32.357233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83688297-dfe4-4910-8e53-a879c7eb3594 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:32.367249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 497/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:32.368218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:32.469281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:32.473639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4787db-93bd-4444-bd6b-87f69016ef77 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:32.508842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac806214-c9f9-4a5f-8288-8dce8aaf712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:32.509470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac806214-c9f9-4a5f-8288-8dce8aaf712a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:32.509700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac806214-c9f9-4a5f-8288-8dce8aaf712a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:32.509971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac806214-c9f9-4a5f-8288-8dce8aaf712a None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:32.510622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac806214-c9f9-4a5f-8288-8dce8aaf712a None admin] RBAC: Adding query filter params (type=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:32.510922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac806214-c9f9-4a5f-8288-8dce8aaf712a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:32.512268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac806214-c9f9-4a5f-8288-8dce8aaf712a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:32.517362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 510/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:33.285185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7c3f3-53a1-4817-8f10-840d62563bd8 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:33.285672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7c3f3-53a1-4817-8f10-840d62563bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:33.285672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7c3f3-53a1-4817-8f10-840d62563bd8 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:33.287137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 498/1008] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:33.292280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac211e1-3df2-42b7-940b-aeb72113c630 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:33.292577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac211e1-3df2-42b7-940b-aeb72113c630 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:33.292852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac211e1-3df2-42b7-940b-aeb72113c630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:33.310917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4ac211e1-3df2-42b7-940b-aeb72113c630 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:33.312747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4ac211e1-3df2-42b7-940b-aeb72113c630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:33.381964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 511/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:33.406318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7c3f3-53a1-4817-8f10-840d62563bd8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:33.455389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:33.510329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7c3f3-53a1-4817-8f10-840d62563bd8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:33.544617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac75c5-a0b1-495d-b7e1-66d3aaadafb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:33.545243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac75c5-a0b1-495d-b7e1-66d3aaadafb8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:33.545523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac75c5-a0b1-495d-b7e1-66d3aaadafb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:33.545823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac75c5-a0b1-495d-b7e1-66d3aaadafb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:33.547073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac75c5-a0b1-495d-b7e1-66d3aaadafb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:33.548563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac75c5-a0b1-495d-b7e1-66d3aaadafb8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:33.550393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 499/1010] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:33.555930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac211e1-3df2-42b7-940b-aeb72113c630 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:33.655164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:33.664489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac211e1-3df2-42b7-940b-aeb72113c630 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:33.700168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:33.700768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:33.701065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:33.701322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:33.702045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:33.703567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:33.733037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfc0c4-af3c-4b3a-b27a-75beeed39fca None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:33.736219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfc0c4-af3c-4b3a-b27a-75beeed39fca None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:33.736219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfc0c4-af3c-4b3a-b27a-75beeed39fca None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:33.736219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 500/1011] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:33.740692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b76be-3bcb-4702-a81b-2361a31941de None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:33.740973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b76be-3bcb-4702-a81b-2361a31941de None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:33.741245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b76be-3bcb-4702-a81b-2361a31941de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:33.759743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f47b76be-3bcb-4702-a81b-2361a31941de None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:33.760922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f47b76be-3bcb-4702-a81b-2361a31941de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:33.787913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 512/1012] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:33 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:33.841896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 501/1013] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:33.865972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:33.981986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e762b-2552-4e77-8bcf-003122a85eae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:34.016631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:34.017262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:34.017598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:34.017872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:34.018488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:34.019860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:34.028545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:34.029523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:34.031069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 513/1014] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:34.036820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b76be-3bcb-4702-a81b-2361a31941de None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:34.082711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:34.137572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b76be-3bcb-4702-a81b-2361a31941de None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:34.172480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:34.173087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:34.173366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:34.173696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:34.174399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:34.174705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:34.176073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:34.185768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 502/1015] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:34.186800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:34.198844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:34.289243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0b739-71f8-434e-a327-181f5496e65b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:34.324550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc67208-6116-42ae-a545-285fc6fa8e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:34.325150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc67208-6116-42ae-a545-285fc6fa8e58 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:34.325425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc67208-6116-42ae-a545-285fc6fa8e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:34.325693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc67208-6116-42ae-a545-285fc6fa8e58 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:34.326318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc67208-6116-42ae-a545-285fc6fa8e58 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:34.326645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc67208-6116-42ae-a545-285fc6fa8e58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:34.328625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc67208-6116-42ae-a545-285fc6fa8e58 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:34.333634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 514/1016] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 02:15:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:34.339078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:34.439492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9b5b8-adf0-4f7e-a387-cddaf7757684 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:34.448818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:34.473973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:34.474861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:34.475370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:34.475800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:34.476622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:34.478199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:34.517811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5190b-11d1-4bb9-bc80-f2ccc4c26c26 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:34.518092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5190b-11d1-4bb9-bc80-f2ccc4c26c26 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:34.518448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5190b-11d1-4bb9-bc80-f2ccc4c26c26 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:34.519783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 515/1017] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:34.524628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9f3e-1ecf-41e2-8076-64c74512e62e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:34.524910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9f3e-1ecf-41e2-8076-64c74512e62e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:34.525184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a9f3e-1ecf-41e2-8076-64c74512e62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:34.543699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fd8a9f3e-1ecf-41e2-8076-64c74512e62e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:34.544938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fd8a9f3e-1ecf-41e2-8076-64c74512e62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:34.564433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 503/1018] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:34.618434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:34.627590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 516/1019] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:34 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:34.650866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:34.762792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7af33-b912-4e1d-b854-6acaf794f642 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:34.798776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ededa-9ed9-4e8f-a0ff-08a028564ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:34.799388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ededa-9ed9-4e8f-a0ff-08a028564ba0 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:34.799650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ededa-9ed9-4e8f-a0ff-08a028564ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:34.799916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ededa-9ed9-4e8f-a0ff-08a028564ba0 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:34.800566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ededa-9ed9-4e8f-a0ff-08a028564ba0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:34.801923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ededa-9ed9-4e8f-a0ff-08a028564ba0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:34.889919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 504/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:34 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:35.859559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a7ce-d9d5-4e66-9c2d-654f8ceb03ff None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:35.860102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a7ce-d9d5-4e66-9c2d-654f8ceb03ff None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:35.860102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a7ce-d9d5-4e66-9c2d-654f8ceb03ff None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:35.861579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 517/1021] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:35.866851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d7e2-6681-436b-b16c-59884c5490a0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:35.867224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d7e2-6681-436b-b16c-59884c5490a0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:35.867395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80d7e2-6681-436b-b16c-59884c5490a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:35.885503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ec80d7e2-6681-436b-b16c-59884c5490a0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:35.886708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ec80d7e2-6681-436b-b16c-59884c5490a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:35.905997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:35.968098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 505/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:35.992601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15a7ce-d9d5-4e66-9c2d-654f8ceb03ff None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:36.096719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15a7ce-d9d5-4e66-9c2d-654f8ceb03ff None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:36.132019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bbbc3-6b31-4098-8d82-36c582910b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:36.132559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bbbc3-6b31-4098-8d82-36c582910b9c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:36.132839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bbbc3-6b31-4098-8d82-36c582910b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:36.133105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bbbc3-6b31-4098-8d82-36c582910b9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:36.134212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bbbc3-6b31-4098-8d82-36c582910b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:36.135660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bbbc3-6b31-4098-8d82-36c582910b9c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:36.137361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 518/1023] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:36.143081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80d7e2-6681-436b-b16c-59884c5490a0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:36.241990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80d7e2-6681-436b-b16c-59884c5490a0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:36.276822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba346d-a933-4853-80ab-6570f5e7b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:36.277426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba346d-a933-4853-80ab-6570f5e7b0d2 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:36.277704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba346d-a933-4853-80ab-6570f5e7b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:36.277975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba346d-a933-4853-80ab-6570f5e7b0d2 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:36.278656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba346d-a933-4853-80ab-6570f5e7b0d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:36.280131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba346d-a933-4853-80ab-6570f5e7b0d2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:36.386416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 506/1024] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:15:36 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:36.665962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2a35f-faae-4319-a84e-770472f8aeab None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:36.666161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2a35f-faae-4319-a84e-770472f8aeab None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:36.666441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2a35f-faae-4319-a84e-770472f8aeab None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:36.667773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 519/1025] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:36.673765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07d1d-c7fb-4478-81a5-d58837bde357 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:36.674049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07d1d-c7fb-4478-81a5-d58837bde357 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:36.674322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07d1d-c7fb-4478-81a5-d58837bde357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:36.687776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:36.693261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-09b07d1d-c7fb-4478-81a5-d58837bde357 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:36.697612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-09b07d1d-c7fb-4478-81a5-d58837bde357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:36.763173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 507/1026] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:36.786700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2a35f-faae-4319-a84e-770472f8aeab None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:36.855172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:36.891062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2a35f-faae-4319-a84e-770472f8aeab None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:36.930859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a31369-1219-46ee-8208-e0d4d1f99274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:36.931485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a31369-1219-46ee-8208-e0d4d1f99274 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:36.931760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a31369-1219-46ee-8208-e0d4d1f99274 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:36.932026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a31369-1219-46ee-8208-e0d4d1f99274 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:36.943172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a31369-1219-46ee-8208-e0d4d1f99274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-82a31369-1219-46ee-8208-e0d4d1f99274 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:36.947724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:36.948656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:36.954916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 520/1027] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 02:15:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:36.955672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b07d1d-c7fb-4478-81a5-d58837bde357 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:36.962647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51476-de7d-4fe9-8b5d-58cd8cd8ceb7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:36.962936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51476-de7d-4fe9-8b5d-58cd8cd8ceb7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:36.963749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e51476-de7d-4fe9-8b5d-58cd8cd8ceb7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:36.966203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 521/1028] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:36.971729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef2c8-e6a9-4d61-a8df-62206b66802e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:36.972016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef2c8-e6a9-4d61-a8df-62206b66802e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:36.972290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef2c8-e6a9-4d61-a8df-62206b66802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:36.990943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2e1ef2c8-e6a9-4d61-a8df-62206b66802e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:36.992141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2e1ef2c8-e6a9-4d61-a8df-62206b66802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:37.061352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b07d1d-c7fb-4478-81a5-d58837bde357 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.061585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 522/1029] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:37.083663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef2c8-e6a9-4d61-a8df-62206b66802e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.099043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.099655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.099937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.100208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.101043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:37.101354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.103188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.105394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.112635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 508/1030] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 02:15:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:37.128865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.151453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.197996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef2c8-e6a9-4d61-a8df-62206b66802e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.222470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5c42b-a292-4310-accf-b6bdd3992893 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.233891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.234617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.234892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.235161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.235743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.237098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:37.246015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:37.247098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:37.253124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 523/1031] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:37.253949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.262780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.263409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.266284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.266284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.275764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.277344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:37.292213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:37.293245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:37.299866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:37.299866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:37.299866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:37.299866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 509/1032] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:37.299866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.349546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.363157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134349f-7b07-491f-828e-abe2200a24d4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.384956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.400735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.401579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.402246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.402547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.403440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] RBAC: Adding query filter params (name=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:37.403738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.404794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d7e79-ac92-463c-bfec-1a839ab78ecc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.405686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.411775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 524/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:37.417261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.445143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.445740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.446016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.446284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.446947 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:37.447244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.462777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 510/1034] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:37.468280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.533769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef28055-f3ee-44a5-ae89-7192df14df9a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.571531 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.572125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.572408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.572754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.573392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] RBAC: Adding query filter params (type=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:37.573696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.575108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.580655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5f873-dfec-4f08-a0e3-7356d7d6bbe7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.580786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 525/1035] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:37.586583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.593855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.617852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.618499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.618776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.619041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.626645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.629397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:37.642157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:37.643160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:37.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 511/1036] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:37.646578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.648913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.700238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc4ad5-c0a5-4483-8dc5-fc4448e4615f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.735361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.735957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.736232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.736499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.737183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] RBAC: Adding query filter params (interface=public, service_id=b8d101d6a8c34a0b8dffddc29c31c04d, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:37.737488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.739168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.744945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 526/1037] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/endpoints?service_id=b8d101d6a8c34a0b8dffddc29c31c04d&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:37.750917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.762018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aad98c-ba93-480e-8e00-6c7743e87b7c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.798285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.798914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.799186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.799453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.800179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:37.800463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.801959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.819535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 512/1038] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:37.825765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:37.854714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.857324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbd51f-59b0-4718-a5bf-5b9bd080547b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.870961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:37.896366 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.896983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.897336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.897602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:37.898307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:37.899741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:37.914399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 527/1039] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:15:37.936043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b489c-298b-400a-bf0b-a0eeb5df64c2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:37.975998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:37.976639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:37.976868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:37.977049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.052209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:38.053663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:38.082411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:38.153932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 513/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:15:38.160024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.260518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b026f-65bc-4a9c-a57b-57d77cac79da None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:38.294076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d1fd-acfd-4363-bafb-d2861b9cfb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:38.294722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d1fd-acfd-4363-bafb-d2861b9cfb69 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.294995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d1fd-acfd-4363-bafb-d2861b9cfb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.295262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d1fd-acfd-4363-bafb-d2861b9cfb69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.306538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d1fd-acfd-4363-bafb-d2861b9cfb69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:38.307971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d1fd-acfd-4363-bafb-d2861b9cfb69 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-97a0d1fd-acfd-4363-bafb-d2861b9cfb69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:38.322465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:38.323432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:38.324328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:38.324328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:38.324328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:38.324328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 528/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:38.330277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.431942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53bb51-65f0-4b32-91ae-71773f49538b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:38.458473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:38.466803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:38.467934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.472534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.472534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.472534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:38.472534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:38.472534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:38.477680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda0e76-bdbb-4d16-be2e-f17712727aa6 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.477977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda0e76-bdbb-4d16-be2e-f17712727aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.478296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda0e76-bdbb-4d16-be2e-f17712727aa6 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.479678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 529/1042] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:38.482512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 514/1043] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:38.487829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991251-f931-4d64-9694-007d1f145e66 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.488111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991251-f931-4d64-9694-007d1f145e66 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.488650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7991251-f931-4d64-9694-007d1f145e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.491435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.506677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e7991251-f931-4d64-9694-007d1f145e66 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:38.507880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e7991251-f931-4d64-9694-007d1f145e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:38.533065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:38.586659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 530/1044] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:38.595408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5d895-d971-4507-a174-62f86babb723 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:38.603557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7991251-f931-4d64-9694-007d1f145e66 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.630905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:38.631543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.631807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.632076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.639523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:38.641115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:38.652404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:38.653356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:38.654253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 515/1045] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:38.659678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.708418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7991251-f931-4d64-9694-007d1f145e66 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:38.714830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:38.748130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:38.749092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.749092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.749372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.759921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:38.764719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:38.765636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:38.766540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 531/1046] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:38.771728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.774839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f2f51-9dd3-4652-b191-624ff60ea5dc None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:38.793938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:38.813289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:38.813964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.814302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.814660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.815404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:38.815770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:38.817248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:38.830521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 516/1047] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:38.836434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.883879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e686c-54fa-428d-9da9-8e15a08a9f6e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:38.926885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:38.927592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.927874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.928164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.928913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:38.929182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:38.930504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:38.939639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 532/1048] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:38.945090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:38.946675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49ec4a-f54e-4cbb-bcef-c3ae68073d80 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:38.957781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:38.983794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:38.984407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:38.984676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:38.984949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:38.985673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] RBAC: Adding query filter params (role.id=8611063528db47a3aba26d87c4f05a9f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=c4cfb26c57514e2cbaa73cb036118dfd) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:38.985981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:38.987451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:38.993935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 517/1049] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/role_assignments?user.id=c4cfb26c57514e2cbaa73cb036118dfd&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=8611063528db47a3aba26d87c4f05a9f => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:39.040890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:39.061757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e654248-d8b3-49fb-9457-c9d5d9d969ed None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:39.101689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:39.102330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:39.102640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:39.102913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] PATH_INFO: `/v3/users/f35ff89cfbac4155bc697d91609f9a85` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:39.118214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] RBAC: Authorizing `identity:get_user(user_id=f35ff89cfbac4155bc697d91609f9a85)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:39.119716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:39.121653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 533/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:38 2022] GET /identity/v3/users/f35ff89cfbac4155bc697d91609f9a85 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:39.127509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:39.237742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71544e70-151d-42bb-a398-b549a0ef782c None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:39.273368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:39.273967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:39.274240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:39.274542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:39.317546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:39.345727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:39.347207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:39.449987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 518/1051] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:39 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:39.456698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:39.559109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b6d75-96d6-48f1-ad70-26f20f2331bc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:39.594885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb3b8-d81c-41f2-9d6d-54e45749e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:39.595422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb3b8-d81c-41f2-9d6d-54e45749e6bb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:39.595695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb3b8-d81c-41f2-9d6d-54e45749e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:39.595960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb3b8-d81c-41f2-9d6d-54e45749e6bb None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:39.597065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb3b8-d81c-41f2-9d6d-54e45749e6bb None admin] RBAC: Authorizing `identity:get_project(project_id=c4ea9384100349ffa4979091c4bc05f2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:39.598509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb3b8-d81c-41f2-9d6d-54e45749e6bb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:39.600276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 534/1052] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:39 2022] GET /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:39.606095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:39.676774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:39.707127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eaa9e-c171-4bb8-8b64-529659e1152b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:39.742545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37d8a5-4867-4330-b028-9c0bc695c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:39.743230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d8a5-4867-4330-b028-9c0bc695c1fb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:39.743572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d8a5-4867-4330-b028-9c0bc695c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:39.743907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d8a5-4867-4330-b028-9c0bc695c1fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:39.744697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37d8a5-4867-4330-b028-9c0bc695c1fb None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=c4ea9384100349ffa4979091c4bc05f2, user.id=f35ff89cfbac4155bc697d91609f9a85) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:39.745066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37d8a5-4867-4330-b028-9c0bc695c1fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:39.746573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37d8a5-4867-4330-b028-9c0bc695c1fb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:39.752697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 519/1053] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:39 2022] GET /identity/v3/role_assignments?user.id=f35ff89cfbac4155bc697d91609f9a85&scope.project.id=c4ea9384100349ffa4979091c4bc05f2&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:39.936993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e346a33-11a7-4a52-be0e-282f6300b288 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:39.937585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e346a33-11a7-4a52-be0e-282f6300b288 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:39.937811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e346a33-11a7-4a52-be0e-282f6300b288 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:39.939211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 535/1054] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:39.944432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd39f-4b4a-4483-8eb6-3911a20fa78f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:39.944715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd39f-4b4a-4483-8eb6-3911a20fa78f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:39.944980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd39f-4b4a-4483-8eb6-3911a20fa78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:39.963008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d28dd39f-4b4a-4483-8eb6-3911a20fa78f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:39.964190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d28dd39f-4b4a-4483-8eb6-3911a20fa78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:40.033195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 520/1055] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:40.056014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e346a33-11a7-4a52-be0e-282f6300b288 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:40.065444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:40.162616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e346a33-11a7-4a52-be0e-282f6300b288 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:40.198906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:40.199450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:40.199730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:40.200005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] PATH_INFO: `/v3/services/image` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:40.200588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:40.202058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:40.210853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:40.211828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:40.212669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 536/1056] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 8 02:15:40 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:40.218468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dd39f-4b4a-4483-8eb6-3911a20fa78f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:40.283794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:40.318681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dd39f-4b4a-4483-8eb6-3911a20fa78f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:40.352816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:40.353411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:40.353685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:40.353951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:40.354590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] RBAC: Adding query filter params (name=image) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:40.354887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:40.356241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:40.361330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 521/1057] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:40.367200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:40.435806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:40.468536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6eb4d5-49bf-4b14-b222-d2ab8440153b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:40.501673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13329-1e05-4a68-92b4-c5990b1de374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:40.502270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13329-1e05-4a68-92b4-c5990b1de374 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:40.502568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13329-1e05-4a68-92b4-c5990b1de374 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:40.502836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13329-1e05-4a68-92b4-c5990b1de374 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:40.503456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c13329-1e05-4a68-92b4-c5990b1de374 None admin] RBAC: Adding query filter params (type=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:40.503754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c13329-1e05-4a68-92b4-c5990b1de374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:40.505097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c13329-1e05-4a68-92b4-c5990b1de374 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:40.510039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 537/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:40.516157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:40.615694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29732a-8f63-4e58-abc7-c602e7fec095 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:40.643877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:40.649921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e74e7-f3f4-46c3-9b81-8dbb4617dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:40.650555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e74e7-f3f4-46c3-9b81-8dbb4617dd73 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:40.650837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e74e7-f3f4-46c3-9b81-8dbb4617dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:40.651219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e74e7-f3f4-46c3-9b81-8dbb4617dd73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:40.656559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e74e7-f3f4-46c3-9b81-8dbb4617dd73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:40.656559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e74e7-f3f4-46c3-9b81-8dbb4617dd73 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:40.735418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 522/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:15:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:41.022608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299334c-dd59-42db-9567-0891e2a1e94d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.023097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299334c-dd59-42db-9567-0891e2a1e94d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.023483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299334c-dd59-42db-9567-0891e2a1e94d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.026069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 538/1060] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:41.033096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9694930-85ef-4e19-8a35-e37fad6a8017 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.033446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9694930-85ef-4e19-8a35-e37fad6a8017 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.033933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9694930-85ef-4e19-8a35-e37fad6a8017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.051684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f9694930-85ef-4e19-8a35-e37fad6a8017 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:41.052863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f9694930-85ef-4e19-8a35-e37fad6a8017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:41.128829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 523/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:41.150382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299334c-dd59-42db-9567-0891e2a1e94d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:41.256866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299334c-dd59-42db-9567-0891e2a1e94d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:41.290636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:41.291258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.291529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.291794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.293644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:41.303335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:41.308016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:41.308959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:41.315300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 539/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 02:15:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:41.318142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9694930-85ef-4e19-8a35-e37fad6a8017 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:41.421766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9694930-85ef-4e19-8a35-e37fad6a8017 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:41.436322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:41.459654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:41.460267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.460542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.460809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.461531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:41.461901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:41.463192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:41.472915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 524/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 02:15:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:41.478120 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:41.579248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f691-9291-404a-9206-50c0abcaf989 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:41.612216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04e14f-0aa7-438b-8701-88fa467b4fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:41.612812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e14f-0aa7-438b-8701-88fa467b4fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.613085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e14f-0aa7-438b-8701-88fa467b4fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.613605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e14f-0aa7-438b-8701-88fa467b4fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.624153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04e14f-0aa7-438b-8701-88fa467b4fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:41.625578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04e14f-0aa7-438b-8701-88fa467b4fe6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2f04e14f-0aa7-438b-8701-88fa467b4fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:41.640509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:41.641431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:41.642367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:41.642367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:41.642367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:41.646370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 540/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:41.647778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:41.746787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2e974-932f-4114-9dac-6fc4816e6852 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:41.770944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f0869-3052-49d5-95b8-e97c6e679a3e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.771221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f0869-3052-49d5-95b8-e97c6e679a3e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.771498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f0869-3052-49d5-95b8-e97c6e679a3e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.773033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 541/1065] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:41.777613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973ccd1-691b-4ccc-af9b-c4cb65ba3be1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.777894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973ccd1-691b-4ccc-af9b-c4cb65ba3be1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.778170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973ccd1-691b-4ccc-af9b-c4cb65ba3be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.781372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:41.781968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.782245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.782541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.783182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:41.783483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:41.784850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:41.795950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9973ccd1-691b-4ccc-af9b-c4cb65ba3be1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:41.796937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 525/1066] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:41.797280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9973ccd1-691b-4ccc-af9b-c4cb65ba3be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:41.802432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:41.837813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:41.841882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:41.868869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 542/1067] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:41.892524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973ccd1-691b-4ccc-af9b-c4cb65ba3be1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:41.908438 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9244e-530d-47d8-b3a5-44232611d946 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:41.946658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:41.947280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:41.947558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:41.947823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:41.955325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:41.956760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:41.968570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:41.974278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:41.975539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 526/1068] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:15:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:41.977750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.019603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973ccd1-691b-4ccc-af9b-c4cb65ba3be1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.054681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.055287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.055565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.055835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.066596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:42.071220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:42.072167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:42.073014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:42.077807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 543/1069] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:42.078665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.079481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.086741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc627a28-a991-4284-95e8-ced5101ff0de None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.114427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.122181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.122803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.125079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.125079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.125079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:42.125079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.126396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.143775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 527/1070] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:15:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:42.149049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.184420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffe647-1ab3-4534-ba9e-1bcfb4b67f0d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.219475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.220047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.220330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.220591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.221243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:42.221537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.222919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.235838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 544/1071] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:42.237681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.252830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d910f-dcf2-46e6-8fd7-dec7470404eb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.291014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.291641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.291924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.292195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.302762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.304296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:42.319965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:42.320958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:42.322447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:42.322447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:42.322447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:42.322447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 528/1072] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:42.326624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.331800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.344616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.349592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb5df7-56ba-4c22-9f8e-a1948b4d965b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.394309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.395210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.395629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.396055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] PATH_INFO: `/v3/users/f35ff89cfbac4155bc697d91609f9a85` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.397412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] RBAC: Authorizing `identity:get_user(user_id=f35ff89cfbac4155bc697d91609f9a85)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.398983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.405398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 545/1073] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/users/f35ff89cfbac4155bc697d91609f9a85 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:42.406503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.435066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d8760-be76-4401-9df5-94f3d3213904 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.471935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.472660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.472936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.473209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.473851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:42.474153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.475668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.493383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 529/1074] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:42.494713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.508504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c203bd-bad8-443b-a504-c2dc08f1cdf3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.543836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.544559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.544838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.545101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.547200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] RBAC: Authorizing `identity:get_project(project_id=c4ea9384100349ffa4979091c4bc05f2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.548608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.550389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 546/1075] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:42.556034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.562750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.594064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.606627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b735de-ee56-4ec0-8972-3e1bf255f7cd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.643192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.643824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.644100 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.644420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.651152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.652598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.663319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3383f8-158a-49b7-97fc-96a2873b2b9e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:42.664260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:42.665117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:42.666472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 530/1076] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:42.671281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.702082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865fe5f-f5a1-444e-ba88-cc0abe064eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.702831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865fe5f-f5a1-444e-ba88-cc0abe064eb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.703116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865fe5f-f5a1-444e-ba88-cc0abe064eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.703384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865fe5f-f5a1-444e-ba88-cc0abe064eb8 None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2/users/f35ff89cfbac4155bc697d91609f9a85/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.705299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865fe5f-f5a1-444e-ba88-cc0abe064eb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ea9384100349ffa4979091c4bc05f2, user_id=f35ff89cfbac4155bc697d91609f9a85, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.706749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865fe5f-f5a1-444e-ba88-cc0abe064eb8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.770192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 547/1077] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:42 2022] PUT /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2/users/f35ff89cfbac4155bc697d91609f9a85/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:15:42.778761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e5d0-382e-4c67-a5e2-4b77303c5f27 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.796307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705ccd-403b-4467-b15b-0fa9f1e6c144 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.796566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705ccd-403b-4467-b15b-0fa9f1e6c144 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.796919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705ccd-403b-4467-b15b-0fa9f1e6c144 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.798215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 548/1078] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:42.803575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.803840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dac69-a0ef-4de6-852f-bee1d6f1baaf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.804123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dac69-a0ef-4de6-852f-bee1d6f1baaf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.804397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088dac69-a0ef-4de6-852f-bee1d6f1baaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.815582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.818566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.818566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.818566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:42.818566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:42.818566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:42.819403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:42.824949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-088dac69-a0ef-4de6-852f-bee1d6f1baaf None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:42.826372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-088dac69-a0ef-4de6-852f-bee1d6f1baaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:42.838285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 531/1079] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:42.839341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.903268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 549/1080] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:42.926600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088dac69-a0ef-4de6-852f-bee1d6f1baaf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:42.954872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:42.957812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58a11-62a0-41f9-887a-b3269b09fb3a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:42.996939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:42.997688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:42.998037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:42.998390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/c4cfb26c57514e2cbaa73cb036118dfd/roles/8611063528db47a3aba26d87c4f05a9f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:43.000745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=c4cfb26c57514e2cbaa73cb036118dfd, role_id=8611063528db47a3aba26d87c4f05a9f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:43.002214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:43.034301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088dac69-a0ef-4de6-852f-bee1d6f1baaf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:43.054760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 532/1081] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:42 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/c4cfb26c57514e2cbaa73cb036118dfd/roles/8611063528db47a3aba26d87c4f05a9f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:15:43.080406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:43.081642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:43.082097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:43.082570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:43.159978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:43.161687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:43.190581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:43.271112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 550/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:43.276856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:43.326302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:43.385154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fd4ca-b758-4017-a61d-99b4a470faa8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:43.419396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6ff03-ae48-463a-837c-0b8698127064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:43.419941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ff03-ae48-463a-837c-0b8698127064 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:43.420798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ff03-ae48-463a-837c-0b8698127064 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:43.421157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ff03-ae48-463a-837c-0b8698127064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:43.431401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6ff03-ae48-463a-837c-0b8698127064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:43.432840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6ff03-ae48-463a-837c-0b8698127064 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-95d6ff03-ae48-463a-837c-0b8698127064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:43.447692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:43.448656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:43.449593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:43.449593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:43.449593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:43.450459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 533/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:43.454885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:43.555856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb91e9-f287-456d-9707-fe48ffee2d04 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:43.579754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:43.590755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdaeeb-672d-477a-937e-8a7c3a0e0947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:43.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdaeeb-672d-477a-937e-8a7c3a0e0947 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:43.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdaeeb-672d-477a-937e-8a7c3a0e0947 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:43.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdaeeb-672d-477a-937e-8a7c3a0e0947 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:43.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdaeeb-672d-477a-937e-8a7c3a0e0947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:43.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdaeeb-672d-477a-937e-8a7c3a0e0947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:43.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdaeeb-672d-477a-937e-8a7c3a0e0947 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:43.606896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 551/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:44.808671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05db463-2f19-4216-9c2b-667760ec6816 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:44.809226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05db463-2f19-4216-9c2b-667760ec6816 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:44.809226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05db463-2f19-4216-9c2b-667760ec6816 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:44.810589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 534/1085] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:44.816398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c88133-cd4e-4ed9-930c-964a4c6554f8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:44.816788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c88133-cd4e-4ed9-930c-964a4c6554f8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:44.816938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c88133-cd4e-4ed9-930c-964a4c6554f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:44.835459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a1c88133-cd4e-4ed9-930c-964a4c6554f8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:44.836618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a1c88133-cd4e-4ed9-930c-964a4c6554f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:44.903819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 552/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:44.927690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05db463-2f19-4216-9c2b-667760ec6816 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.020998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.027905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05db463-2f19-4216-9c2b-667760ec6816 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.058279 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e7ba5-d73b-49be-b618-577c6a7897da None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.058591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e7ba5-d73b-49be-b618-577c6a7897da None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.058919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e7ba5-d73b-49be-b618-577c6a7897da None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.060199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 553/1087] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:15:45.062418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.063051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.063333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.063606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.065897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab6cc4-6b74-4e89-9590-2088386cd553 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.066175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab6cc4-6b74-4e89-9590-2088386cd553 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.066476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab6cc4-6b74-4e89-9590-2088386cd553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.074246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:45.079013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:45.080172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:15:45.081031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 535/1088] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:15:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:45.085909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.086917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a9ab6cc4-6b74-4e89-9590-2088386cd553 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:45.088106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a9ab6cc4-6b74-4e89-9590-2088386cd553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:45.154192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 554/1089] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:15:45.177277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab6cc4-6b74-4e89-9590-2088386cd553 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.186288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a579f4-aa5d-4514-9ec0-c32e0d37b329 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.219733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.220332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.220612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.220885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.221544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:45.221846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.223322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.223666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.231404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 536/1090] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:45.237903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.256776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.278282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab6cc4-6b74-4e89-9590-2088386cd553 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.314002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.314581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.314887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.315158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.325160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:15:45.329846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:45.330996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:45.332108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 555/1091] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:45.336979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.339450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a9b79-3743-40dc-a043-6ce3ce018348 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.373873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.374534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.374816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.375087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] PATH_INFO: `/v3/users/f35ff89cfbac4155bc697d91609f9a85` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.376216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] RBAC: Authorizing `identity:get_user(user_id=f35ff89cfbac4155bc697d91609f9a85)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.377646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.379242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 537/1092] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/users/f35ff89cfbac4155bc697d91609f9a85 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:45.384954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.434685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f343a6-45a8-44fc-9249-fd7659b4ae8f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.437441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.468103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.469365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.469962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.470243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.470552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.471218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:45.471522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.472891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.480877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 556/1093] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:45.483253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35291ea-9a11-4627-8237-b8d15bf4e0cb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.487000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.517178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.517774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.518047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.518314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.583850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.585005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94e772-0ed8-454d-8376-e094b57e4a41 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.585217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.618927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.620234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.620508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.620831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.631363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.632784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:45.647309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:45.648290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:45.649173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:45.649173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:45.649173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:45.649173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 557/1094] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:45.653980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.672198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.673379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.687770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 538/1095] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:15:45.693549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.753019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfde85-b44d-409a-b100-b9fe3f658eb1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.787683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.788281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.788560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.788833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.789464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:45.789767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.790773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ecba-57d3-441d-bbda-4ea3567b663e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.791148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.802392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 558/1096] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:45.808241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.827644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.828268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.828546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.828820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.829942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] RBAC: Authorizing `identity:get_project(project_id=c4ea9384100349ffa4979091c4bc05f2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.831396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.833025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 539/1097] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:45.839001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.886867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.893653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:45.908091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678c59a-91fc-4583-9b96-58906de38d65 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.937942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c3e54-b16c-4d22-b58f-cfde9ab5157e None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:45.944460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.945080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.945359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.945630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.954148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.955658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:15:45.967561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:45.968508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:45.969451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 559/1098] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:45.973671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:45.974273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:45.974487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:45.974601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:45.974852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:45.975576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=c4ea9384100349ffa4979091c4bc05f2, user.id=f35ff89cfbac4155bc697d91609f9a85) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:45.975882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:45.977304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:45.983660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 540/1099] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/role_assignments?user.id=f35ff89cfbac4155bc697d91609f9a85&scope.project.id=c4ea9384100349ffa4979091c4bc05f2&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:46.081017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9828e0-2ec0-4bad-a641-009404f3fabc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:46.117278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:46.117798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:46.118076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:46.118376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:46.119111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:46.119418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:46.120854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:46.130118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:46.139873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 560/1100] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 02:15:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:15:46.145720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:46.248108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26c6c-2f38-4e1a-9a1f-d93cf10dbb8b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:46.265033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:46.281970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:46.282599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:46.282878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:46.283142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:46.350383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:46.351820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:46.447944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 541/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:46.454395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:46.552827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f043a-6ad2-4839-8bce-308269dfafe3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:46.586252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:46.586888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:46.587162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:46.587428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:46.597111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:46.598554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:15:46.613399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:46.614658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:15:46.615786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:15:46.615786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:15:46.615786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:46.618492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 561/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:15:46.620222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:46.639075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:46.717704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d51c7-0105-4718-9110-3fe69299a146 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:46.750936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:46.751531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:46.751810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:46.752078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:46.752706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:46.753004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:46.754435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:46.765446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 542/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:46.771109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:46.819098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:46.868483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528152c5-2260-4953-9e40-ce08c5079914 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:46.900087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:46.900685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:46.900943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:46.901207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:46.908091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:46.909498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:15:46.922509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:15:46.923717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:15:46.924815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 562/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:15:46.927717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:47.004652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:47.025023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85930b7-b03b-4834-9af6-60734d4e7a20 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:47.058968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff8ca1-9405-4318-8091-c67ab9b1e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:47.059555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8ca1-9405-4318-8091-c67ab9b1e6bd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:47.059826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8ca1-9405-4318-8091-c67ab9b1e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:47.060092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8ca1-9405-4318-8091-c67ab9b1e6bd None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:47.060968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff8ca1-9405-4318-8091-c67ab9b1e6bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:47.061274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff8ca1-9405-4318-8091-c67ab9b1e6bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:47.062717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff8ca1-9405-4318-8091-c67ab9b1e6bd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:47.074421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 543/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:47.081437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:47.184339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111691fa-83ea-490b-8cb9-384daff41815 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:47.193667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:47.221065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fcd84-6787-44ce-82a0-7636c80b7f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:47.221649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcd84-6787-44ce-82a0-7636c80b7f03 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:47.221924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcd84-6787-44ce-82a0-7636c80b7f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:47.222191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcd84-6787-44ce-82a0-7636c80b7f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:47.222931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fcd84-6787-44ce-82a0-7636c80b7f03 None admin] RBAC: Adding query filter params (role.id=8611063528db47a3aba26d87c4f05a9f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=c4cfb26c57514e2cbaa73cb036118dfd) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:47.223235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fcd84-6787-44ce-82a0-7636c80b7f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:47.224644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fcd84-6787-44ce-82a0-7636c80b7f03 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:47.230610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 563/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:47 2022] GET /identity/v3/role_assignments?user.id=c4cfb26c57514e2cbaa73cb036118dfd&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=8611063528db47a3aba26d87c4f05a9f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:48.027150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c30f6-e22c-4d8a-a2b3-ff1abfe5a3f3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:48.027689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c30f6-e22c-4d8a-a2b3-ff1abfe5a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:48.027689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c30f6-e22c-4d8a-a2b3-ff1abfe5a3f3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:48.029040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 544/1107] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:48.035081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500a4cd-6fae-461d-acbb-52bbbf201f86 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:48.035514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500a4cd-6fae-461d-acbb-52bbbf201f86 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:48.035657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500a4cd-6fae-461d-acbb-52bbbf201f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:48.053875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a500a4cd-6fae-461d-acbb-52bbbf201f86 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:48.055098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a500a4cd-6fae-461d-acbb-52bbbf201f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:48.120071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 564/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:48.148605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c30f6-e22c-4d8a-a2b3-ff1abfe5a3f3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:48.250622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c30f6-e22c-4d8a-a2b3-ff1abfe5a3f3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:48.283225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18053ec-aee3-4d72-aa1a-32ba9dfefd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:48.283830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18053ec-aee3-4d72-aa1a-32ba9dfefd27 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:48.284103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18053ec-aee3-4d72-aa1a-32ba9dfefd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:48.284369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18053ec-aee3-4d72-aa1a-32ba9dfefd27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:48.285451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18053ec-aee3-4d72-aa1a-32ba9dfefd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:48.286968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18053ec-aee3-4d72-aa1a-32ba9dfefd27 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:48.288503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 545/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:48.294294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500a4cd-6fae-461d-acbb-52bbbf201f86 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:48.390712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500a4cd-6fae-461d-acbb-52bbbf201f86 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:48.392179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:48.427688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828273-1deb-4a5e-8407-084f7de36ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:48.428276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828273-1deb-4a5e-8407-084f7de36ec8 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:48.428552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828273-1deb-4a5e-8407-084f7de36ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:48.428821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828273-1deb-4a5e-8407-084f7de36ec8 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:48.429463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea828273-1deb-4a5e-8407-084f7de36ec8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:48.430901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea828273-1deb-4a5e-8407-084f7de36ec8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:48.516781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 565/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:15:48 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:15:50.517338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456a52-4e3c-4080-bceb-decb62ae2996 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:50.517873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456a52-4e3c-4080-bceb-decb62ae2996 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:50.517873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f456a52-4e3c-4080-bceb-decb62ae2996 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:50.519256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 546/1111] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:50.524419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433d7fe-31a0-430d-99c4-6cd0fa84150f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:50.524818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433d7fe-31a0-430d-99c4-6cd0fa84150f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:50.524958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433d7fe-31a0-430d-99c4-6cd0fa84150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:50.543174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b433d7fe-31a0-430d-99c4-6cd0fa84150f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:50.544340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b433d7fe-31a0-430d-99c4-6cd0fa84150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:50.607489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 566/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:50.630195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456a52-4e3c-4080-bceb-decb62ae2996 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:50.639008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:50.730698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f456a52-4e3c-4080-bceb-decb62ae2996 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:50.763121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:50.763726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:50.764005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:50.764272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] PATH_INFO: `/v3/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:50.766280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] RBAC: Authorizing `identity:get_role(role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:50.767739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:50.768980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 547/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:50 2022] GET /identity/v3/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:50.774780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433d7fe-31a0-430d-99c4-6cd0fa84150f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:50.858990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:50.870642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433d7fe-31a0-430d-99c4-6cd0fa84150f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:50.904209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:50.904815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:50.905091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:50.905358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] PATH_INFO: `/v3/users/c96897d6243148c78182df9a9ca6d5f5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:50.918725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] RBAC: Authorizing `identity:get_user(user_id=c96897d6243148c78182df9a9ca6d5f5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:50.920157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:50.922092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 567/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:50 2022] GET /identity/v3/users/c96897d6243148c78182df9a9ca6d5f5 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:50.927521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:51.000319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:51.024249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ab254-98e8-40ab-984b-4a4d0ffcffd4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:51.056022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:51.056608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:51.056883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:51.057151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:51.121622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:51.123081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:51.192405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:51.216651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 548/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:51.222677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:51.366991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f565c90-2ee5-433f-8983-24dc9df0b86f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:51.400216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e35a0-9fd1-41b7-813b-cf2bddf84c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:51.400820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e35a0-9fd1-41b7-813b-cf2bddf84c5b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:51.401097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e35a0-9fd1-41b7-813b-cf2bddf84c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:51.401365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e35a0-9fd1-41b7-813b-cf2bddf84c5b None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:51.402495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e35a0-9fd1-41b7-813b-cf2bddf84c5b None admin] RBAC: Authorizing `identity:get_project(project_id=c4ea9384100349ffa4979091c4bc05f2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:51.403912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e35a0-9fd1-41b7-813b-cf2bddf84c5b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:51.405505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 568/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:51 2022] GET /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:51.411909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:51.528984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7707-8d9c-4cc4-bdba-5bb97ef9ddf1 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:51.562302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720ef63-a545-4762-b7c0-30fd159253af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:51.563695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720ef63-a545-4762-b7c0-30fd159253af None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:51.563978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720ef63-a545-4762-b7c0-30fd159253af None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:51.564248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720ef63-a545-4762-b7c0-30fd159253af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:51.564961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720ef63-a545-4762-b7c0-30fd159253af None admin] RBAC: Adding query filter params (role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=c4ea9384100349ffa4979091c4bc05f2, user.id=c96897d6243148c78182df9a9ca6d5f5) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:51.565270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720ef63-a545-4762-b7c0-30fd159253af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:51.566749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720ef63-a545-4762-b7c0-30fd159253af None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:51.571937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 549/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:51 2022] GET /identity/v3/role_assignments?user.id=c96897d6243148c78182df9a9ca6d5f5&scope.project.id=c4ea9384100349ffa4979091c4bc05f2&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:53.573542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e72b-c1e8-4aa1-ab94-c643981d99d9 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:53.574063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e72b-c1e8-4aa1-ab94-c643981d99d9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:53.574063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782e72b-c1e8-4aa1-ab94-c643981d99d9 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:53.575438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 569/1118] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:53.581271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618ceb3-f92b-4c2d-9131-a716d3b6e35b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:53.581665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618ceb3-f92b-4c2d-9131-a716d3b6e35b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:53.581817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618ceb3-f92b-4c2d-9131-a716d3b6e35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:53.600032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0618ceb3-f92b-4c2d-9131-a716d3b6e35b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:53.601192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0618ceb3-f92b-4c2d-9131-a716d3b6e35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:53.629400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:53.665980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 550/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:53.687400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e72b-c1e8-4aa1-ab94-c643981d99d9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:53.702716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:53.788947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782e72b-c1e8-4aa1-ab94-c643981d99d9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:53.822166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:53.822800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:53.823077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:53.823343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] PATH_INFO: `/v3/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:53.824765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] RBAC: Authorizing `identity:get_role(role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:53.826062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:53.827375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 570/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:53 2022] GET /identity/v3/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:53.833744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618ceb3-f92b-4c2d-9131-a716d3b6e35b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:53.928920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618ceb3-f92b-4c2d-9131-a716d3b6e35b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:53.963328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:53.963967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:53.964242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:53.964511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] PATH_INFO: `/v3/users/c96897d6243148c78182df9a9ca6d5f5` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:53.965618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] RBAC: Authorizing `identity:get_user(user_id=c96897d6243148c78182df9a9ca6d5f5)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:53.967073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:53.969025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 551/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:53 2022] GET /identity/v3/users/c96897d6243148c78182df9a9ca6d5f5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:53.974558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:54.053922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:54.071453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d1802-2f7e-4c75-ad52-d3ebfdd26942 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:54.105114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de0db0-7a2e-4176-bc5d-6d9940b53088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:54.105657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de0db0-7a2e-4176-bc5d-6d9940b53088 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:54.105934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de0db0-7a2e-4176-bc5d-6d9940b53088 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:54.106202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de0db0-7a2e-4176-bc5d-6d9940b53088 None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:54.107325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de0db0-7a2e-4176-bc5d-6d9940b53088 None admin] RBAC: Authorizing `identity:get_project(project_id=c4ea9384100349ffa4979091c4bc05f2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:54.108749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de0db0-7a2e-4176-bc5d-6d9940b53088 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:54.110511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 571/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:53 2022] GET /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:54.116688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:54.150791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:54.211850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcbf1b-c2c3-4c44-8098-b6e71ad20929 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:54.245232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de397d52-e11f-42ae-bf3d-7239ced25b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:54.245873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de397d52-e11f-42ae-bf3d-7239ced25b11 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:54.246149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de397d52-e11f-42ae-bf3d-7239ced25b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:54.246448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de397d52-e11f-42ae-bf3d-7239ced25b11 None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2/users/c96897d6243148c78182df9a9ca6d5f5/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:54.248557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de397d52-e11f-42ae-bf3d-7239ced25b11 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ea9384100349ffa4979091c4bc05f2, user_id=c96897d6243148c78182df9a9ca6d5f5, role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:54.250723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de397d52-e11f-42ae-bf3d-7239ced25b11 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:54.356153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 552/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:15:54 2022] PUT /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2/users/c96897d6243148c78182df9a9ca6d5f5/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:15:56.303639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc215-09b7-4da5-b0bb-e3f5d7aa6a07 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:56.304173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc215-09b7-4da5-b0bb-e3f5d7aa6a07 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:56.304173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc215-09b7-4da5-b0bb-e3f5d7aa6a07 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:56.305494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 572/1124] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:56.310591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a044b2-520a-4c4b-88bb-fd73d57fd271 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:56.311006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a044b2-520a-4c4b-88bb-fd73d57fd271 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:56.311143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a044b2-520a-4c4b-88bb-fd73d57fd271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:56.328877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e6a044b2-520a-4c4b-88bb-fd73d57fd271 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:56.330049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e6a044b2-520a-4c4b-88bb-fd73d57fd271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:56.394844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 553/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:56.417582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc215-09b7-4da5-b0bb-e3f5d7aa6a07 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:56.514444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc215-09b7-4da5-b0bb-e3f5d7aa6a07 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:56.547349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:56.547946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:56.548217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:56.548481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] PATH_INFO: `/v3/roles/a1a80b6188e748c5b83b2718d9a4729e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:56.549831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] RBAC: Authorizing `identity:get_role(role_id=a1a80b6188e748c5b83b2718d9a4729e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:56.551189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:56.552448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 573/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:56 2022] GET /identity/v3/roles/a1a80b6188e748c5b83b2718d9a4729e => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:56.558112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a044b2-520a-4c4b-88bb-fd73d57fd271 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:56.584338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:56.654214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a044b2-520a-4c4b-88bb-fd73d57fd271 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:56.686916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:56.687578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:56.687861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:56.688132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] PATH_INFO: `/v3/users/c96897d6243148c78182df9a9ca6d5f5` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:56.689246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] RBAC: Authorizing `identity:get_user(user_id=c96897d6243148c78182df9a9ca6d5f5)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:56.690700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:56.692250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 554/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:15:56 2022] GET /identity/v3/users/c96897d6243148c78182df9a9ca6d5f5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:56.697835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:56.717277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:56.797906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ce3f-1679-4980-aebd-19df085d40a8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:56.832234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:56.832751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:56.833023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:56.833286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:56.899363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:56.900792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:56.920908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:56.996876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 574/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:15:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:15:57.003430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:57.093255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:57.099625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fcc961-eb36-481b-9825-a03124b0fbc4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:57.133496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d36c4e-3568-419a-8537-122ff3747059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:57.134100 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d36c4e-3568-419a-8537-122ff3747059 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:57.134397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d36c4e-3568-419a-8537-122ff3747059 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:57.134671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d36c4e-3568-419a-8537-122ff3747059 None admin] PATH_INFO: `/v3/projects/c4ea9384100349ffa4979091c4bc05f2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:57.135767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d36c4e-3568-419a-8537-122ff3747059 None admin] RBAC: Authorizing `identity:get_project(project_id=c4ea9384100349ffa4979091c4bc05f2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:57.137191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d36c4e-3568-419a-8537-122ff3747059 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:57.138812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 555/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:15:57 2022] GET /identity/v3/projects/c4ea9384100349ffa4979091c4bc05f2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:57.145305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:57.239844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dd27d-9642-46ab-bd41-7a2295a14b69 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:57.256293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:57.272372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ec9ec-4696-41ed-8b38-f199db163ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:57.273015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec9ec-4696-41ed-8b38-f199db163ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:57.273289 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec9ec-4696-41ed-8b38-f199db163ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:57.273556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ec9ec-4696-41ed-8b38-f199db163ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:57.274270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ec9ec-4696-41ed-8b38-f199db163ca0 None admin] RBAC: Adding query filter params (role.id=a1a80b6188e748c5b83b2718d9a4729e, scope.project.id=c4ea9384100349ffa4979091c4bc05f2, user.id=c96897d6243148c78182df9a9ca6d5f5) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:15:57.274629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ec9ec-4696-41ed-8b38-f199db163ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:57.276015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ec9ec-4696-41ed-8b38-f199db163ca0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:57.281613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 575/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:15:57 2022] GET /identity/v3/role_assignments?user.id=c96897d6243148c78182df9a9ca6d5f5&scope.project.id=c4ea9384100349ffa4979091c4bc05f2&role.id=a1a80b6188e748c5b83b2718d9a4729e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:59.277894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011a2f-59f8-4a12-ba0e-96059d3ba66a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:59.277894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011a2f-59f8-4a12-ba0e-96059d3ba66a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:59.277894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011a2f-59f8-4a12-ba0e-96059d3ba66a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:59.278604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 556/1131] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:15:59.284120 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea51f6-b377-441d-85bd-9996e08f0152 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:59.284487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea51f6-b377-441d-85bd-9996e08f0152 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:59.284665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea51f6-b377-441d-85bd-9996e08f0152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:59.303508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-87ea51f6-b377-441d-85bd-9996e08f0152 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:15:59.304682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-87ea51f6-b377-441d-85bd-9996e08f0152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:15:59.369449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 576/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:15:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:15:59.393627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011a2f-59f8-4a12-ba0e-96059d3ba66a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:59.491488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011a2f-59f8-4a12-ba0e-96059d3ba66a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:59.524103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9d8a5-1471-4339-89b2-b4720eadd9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:59.524727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9d8a5-1471-4339-89b2-b4720eadd9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:59.525007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9d8a5-1471-4339-89b2-b4720eadd9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:59.525281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9d8a5-1471-4339-89b2-b4720eadd9a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:59.526407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9d8a5-1471-4339-89b2-b4720eadd9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:59.527846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9d8a5-1471-4339-89b2-b4720eadd9a3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:59.529450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 557/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:15:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:15:59.535220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea51f6-b377-441d-85bd-9996e08f0152 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:15:59.630807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea51f6-b377-441d-85bd-9996e08f0152 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:15:59.643356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:15:59.663244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56746612-9cf8-4a32-b03d-100a806606cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:15:59.663872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746612-9cf8-4a32-b03d-100a806606cf None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:15:59.664148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746612-9cf8-4a32-b03d-100a806606cf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:15:59.664416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56746612-9cf8-4a32-b03d-100a806606cf None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:15:59.665058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56746612-9cf8-4a32-b03d-100a806606cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:15:59.666496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56746612-9cf8-4a32-b03d-100a806606cf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:15:59.740689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 577/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:15:59 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:16:01.769623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e0eca-c969-4bb0-9b35-3dad7e9a9588 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:01.770203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e0eca-c969-4bb0-9b35-3dad7e9a9588 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:01.770203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e0eca-c969-4bb0-9b35-3dad7e9a9588 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:01.771660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 558/1135] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:01.776924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3136-b73c-4607-a165-ff375ce9f5fa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:01.777304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3136-b73c-4607-a165-ff375ce9f5fa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:01.777474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea3136-b73c-4607-a165-ff375ce9f5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:01.795048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-79ea3136-b73c-4607-a165-ff375ce9f5fa None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:01.796803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-79ea3136-b73c-4607-a165-ff375ce9f5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:01.861495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 578/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:01.885963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e0eca-c969-4bb0-9b35-3dad7e9a9588 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:01.915001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:01.985129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e0eca-c969-4bb0-9b35-3dad7e9a9588 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:02.018502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de32d57-f9f7-471f-a3ff-0dd16bab12ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:02.019109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32d57-f9f7-471f-a3ff-0dd16bab12ec None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:02.019381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32d57-f9f7-471f-a3ff-0dd16bab12ec None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:02.019693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32d57-f9f7-471f-a3ff-0dd16bab12ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:02.020730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de32d57-f9f7-471f-a3ff-0dd16bab12ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:02.022140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de32d57-f9f7-471f-a3ff-0dd16bab12ec None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:02.023702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 559/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:16:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:02.029442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea3136-b73c-4607-a165-ff375ce9f5fa None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:02.120533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:02.126900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea3136-b73c-4607-a165-ff375ce9f5fa None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:02.161083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0541c-52f9-42f5-b255-707b25ade967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:02.166683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0541c-52f9-42f5-b255-707b25ade967 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:02.166683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0541c-52f9-42f5-b255-707b25ade967 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:02.166683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0541c-52f9-42f5-b255-707b25ade967 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:02.166683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0541c-52f9-42f5-b255-707b25ade967 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:02.166683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0541c-52f9-42f5-b255-707b25ade967 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:02.241886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 579/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:16:02 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:16:04.296643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3a23-9f47-4cff-b36a-21cf277777a6 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:04.297175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3a23-9f47-4cff-b36a-21cf277777a6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:04.297175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3a23-9f47-4cff-b36a-21cf277777a6 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:04.298606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 560/1139] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:04.303791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635644-0391-4232-8abe-8b56277860e8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:04.304218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635644-0391-4232-8abe-8b56277860e8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:04.304316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635644-0391-4232-8abe-8b56277860e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:04.323390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-37635644-0391-4232-8abe-8b56277860e8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:04.324572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-37635644-0391-4232-8abe-8b56277860e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:04.394158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 580/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:04.418904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e3a23-9f47-4cff-b36a-21cf277777a6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:04.517556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:04.523156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e3a23-9f47-4cff-b36a-21cf277777a6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:04.564644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:04.565352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:04.565836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:04.566279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:04.579923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:04.585270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:04.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:16:04.592201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 561/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:04.593119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635644-0391-4232-8abe-8b56277860e8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:04.682669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:04.703203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635644-0391-4232-8abe-8b56277860e8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:04.741767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:04.742475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:04.742762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:04.743044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:04.743711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:04.744017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:04.745381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:04.754757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 581/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:16:04.760695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:04.866058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e62f0-e05f-4cc1-ac4d-3dc6b2846e65 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:04.899712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:04.900336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:04.900613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:04.900884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] PATH_INFO: `/v3/users/f022c2f6de1c4cda93e879b208336fd9` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:04.914660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] RBAC: Authorizing `identity:get_user(user_id=f022c2f6de1c4cda93e879b208336fd9)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:04.916132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:04.918927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 562/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:16:04 2022] GET /identity/v3/users/f022c2f6de1c4cda93e879b208336fd9 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:04.924356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:05.039257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e8e59-0a92-42b5-9c93-6a5149edfe2b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:05.073110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:05.073705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:05.073986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:05.074260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:05.085888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:05.140070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:05.141566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:05.235847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 582/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:05.242167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:05.260049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:05.340057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40c33b-85d2-4d21-b398-bcb68fc3844d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:05.380464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f4d40-1b69-4b73-85ae-5158c7610c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:05.381076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f4d40-1b69-4b73-85ae-5158c7610c0b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:05.381353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f4d40-1b69-4b73-85ae-5158c7610c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:05.381619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f4d40-1b69-4b73-85ae-5158c7610c0b None admin] PATH_INFO: `/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:05.384117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f4d40-1b69-4b73-85ae-5158c7610c0b None admin] RBAC: Authorizing `identity:get_project(project_id=e4a1e6b6e52e4a96a0b326d60e5a880d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:05.385613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f4d40-1b69-4b73-85ae-5158c7610c0b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:05.387306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 563/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:16:05 2022] GET /identity/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:05.393555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:05.435896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:05.491077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0b357-cf77-4a28-b7c8-e51ed2ed697e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:05.523823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb5232-3014-4e44-8ea1-5fee25d17d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:05.524406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb5232-3014-4e44-8ea1-5fee25d17d8e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:05.524684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb5232-3014-4e44-8ea1-5fee25d17d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:05.524956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb5232-3014-4e44-8ea1-5fee25d17d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:05.525674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb5232-3014-4e44-8ea1-5fee25d17d8e None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=e4a1e6b6e52e4a96a0b326d60e5a880d, user.id=f022c2f6de1c4cda93e879b208336fd9) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:05.525980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb5232-3014-4e44-8ea1-5fee25d17d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:05.527430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb5232-3014-4e44-8ea1-5fee25d17d8e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:05.533236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 583/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:05 2022] GET /identity/v3/role_assignments?user.id=f022c2f6de1c4cda93e879b208336fd9&scope.project.id=e4a1e6b6e52e4a96a0b326d60e5a880d&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:07.483244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3090f9-d34c-4bbb-8d5f-7583388d777a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:07.483775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3090f9-d34c-4bbb-8d5f-7583388d777a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:07.483898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3090f9-d34c-4bbb-8d5f-7583388d777a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:07.485230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 564/1147] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:16:07.490713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb9015-7105-42cf-a0a0-e82c5126eba4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:07.491123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb9015-7105-42cf-a0a0-e82c5126eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:07.491254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb9015-7105-42cf-a0a0-e82c5126eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:07.509360 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-49eb9015-7105-42cf-a0a0-e82c5126eba4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:07.510564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-49eb9015-7105-42cf-a0a0-e82c5126eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:07.575683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 584/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:07.598165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3090f9-d34c-4bbb-8d5f-7583388d777a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:07.679928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:07.695867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3090f9-d34c-4bbb-8d5f-7583388d777a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:07.728602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:07.729207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:07.729481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:07.729747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:07.740439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:07.744962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:07.746047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:16:07.747217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 565/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:07.752653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb9015-7105-42cf-a0a0-e82c5126eba4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:07.785636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:07.848070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb9015-7105-42cf-a0a0-e82c5126eba4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:07.881821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:07.882430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:07.882709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:07.882978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:07.883629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:07.883927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:07.885270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:07.893250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 585/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:07.899153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:07.995969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800f9b-a178-4932-962a-e049196b2ba4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:08.028559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:08.029172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:08.029448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:08.029717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] PATH_INFO: `/v3/users/f022c2f6de1c4cda93e879b208336fd9` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:08.030956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=f022c2f6de1c4cda93e879b208336fd9)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:08.032393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:08.033946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 566/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:16:07 2022] GET /identity/v3/users/f022c2f6de1c4cda93e879b208336fd9 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:08.039725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:08.136182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:08.140077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc87c0-5c54-47e0-b14c-bf526946cb30 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:08.173838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26cbda-2d00-4221-bc3d-cad0aa00a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:08.174479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26cbda-2d00-4221-bc3d-cad0aa00a749 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:08.174760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26cbda-2d00-4221-bc3d-cad0aa00a749 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:08.175032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26cbda-2d00-4221-bc3d-cad0aa00a749 None admin] PATH_INFO: `/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:08.176142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26cbda-2d00-4221-bc3d-cad0aa00a749 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a1e6b6e52e4a96a0b326d60e5a880d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:08.177551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26cbda-2d00-4221-bc3d-cad0aa00a749 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:08.179176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 586/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:16:08 2022] GET /identity/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:08.184986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:08.200414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:08.283595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9edd34-9019-4f28-80fc-96b441944cf7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:08.315777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c582b-6999-4f82-bd68-d981d0047821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:08.316413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c582b-6999-4f82-bd68-d981d0047821 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:08.316690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c582b-6999-4f82-bd68-d981d0047821 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:08.316960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c582b-6999-4f82-bd68-d981d0047821 None admin] PATH_INFO: `/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d/users/f022c2f6de1c4cda93e879b208336fd9/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:08.319016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c582b-6999-4f82-bd68-d981d0047821 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a1e6b6e52e4a96a0b326d60e5a880d, user_id=f022c2f6de1c4cda93e879b208336fd9, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:08.320448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c582b-6999-4f82-bd68-d981d0047821 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:08.407587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 567/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:16:08 2022] PUT /identity/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d/users/f022c2f6de1c4cda93e879b208336fd9/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:16:10.385145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed0048-69e9-4ddd-8b36-f5a07ec930b8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:10.385691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed0048-69e9-4ddd-8b36-f5a07ec930b8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:10.385691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed0048-69e9-4ddd-8b36-f5a07ec930b8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:10.387040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 587/1154] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:10.392651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610b97-d29f-4fc1-a691-1cd1d11be787 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:10.393039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610b97-d29f-4fc1-a691-1cd1d11be787 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:10.393201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610b97-d29f-4fc1-a691-1cd1d11be787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:10.411230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-01610b97-d29f-4fc1-a691-1cd1d11be787 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:10.412440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-01610b97-d29f-4fc1-a691-1cd1d11be787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:10.476014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 568/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:10.499301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed0048-69e9-4ddd-8b36-f5a07ec930b8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:10.598837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed0048-69e9-4ddd-8b36-f5a07ec930b8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:10.632064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:10.632670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:10.632946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:10.633213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:10.643548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:10.648073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:10.649170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:16:10.650340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 588/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:10.655133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610b97-d29f-4fc1-a691-1cd1d11be787 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:10.657349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:10.752343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610b97-d29f-4fc1-a691-1cd1d11be787 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:10.784152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:10.784739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:10.785016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:10.785282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:10.785934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:10.786232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:10.787621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:10.795260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 569/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:10.801392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:10.828727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:10.898077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c070b69-94a3-4c3e-a7ea-21f1a20da198 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:10.931803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:10.932422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:10.932702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:10.932976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] PATH_INFO: `/v3/users/f022c2f6de1c4cda93e879b208336fd9` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:10.934087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] RBAC: Authorizing `identity:get_user(user_id=f022c2f6de1c4cda93e879b208336fd9)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:10.935546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:10.937895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 589/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:16:10 2022] GET /identity/v3/users/f022c2f6de1c4cda93e879b208336fd9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:10.943571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:11.003971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:11.040330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6dcd9-71f1-4cc4-89dd-b77859bcded8 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:11.073065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:11.073646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:11.073922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:11.074193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:11.138499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:11.140070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:11.197178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:11.234587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 570/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:11.240739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:11.330497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:11.336987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d92dc0-4ef2-48c9-9f11-526cadf2f786 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:11.371083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c3161-fd02-4cbc-b442-84bb3358ba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:11.371700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c3161-fd02-4cbc-b442-84bb3358ba05 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:11.371980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c3161-fd02-4cbc-b442-84bb3358ba05 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:11.372259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c3161-fd02-4cbc-b442-84bb3358ba05 None admin] PATH_INFO: `/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:11.373365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c3161-fd02-4cbc-b442-84bb3358ba05 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a1e6b6e52e4a96a0b326d60e5a880d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:11.374956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c3161-fd02-4cbc-b442-84bb3358ba05 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:11.376550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 590/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:16:11 2022] GET /identity/v3/projects/e4a1e6b6e52e4a96a0b326d60e5a880d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:11.382679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:11.479783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3feb6f-42a2-44ea-ad9c-8d38e51a32a6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:11.512872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8614f-6473-48da-8106-9c0747e6970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:11.513456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8614f-6473-48da-8106-9c0747e6970e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:11.513730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8614f-6473-48da-8106-9c0747e6970e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:11.513993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8614f-6473-48da-8106-9c0747e6970e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:11.515362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8614f-6473-48da-8106-9c0747e6970e None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=e4a1e6b6e52e4a96a0b326d60e5a880d, user.id=f022c2f6de1c4cda93e879b208336fd9) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:11.515661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8614f-6473-48da-8106-9c0747e6970e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:11.517077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8614f-6473-48da-8106-9c0747e6970e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:11.522581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 571/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:11 2022] GET /identity/v3/role_assignments?user.id=f022c2f6de1c4cda93e879b208336fd9&scope.project.id=e4a1e6b6e52e4a96a0b326d60e5a880d&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:13.525617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ed5d3-7a19-462b-a47e-2b1dd921eb0e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:13.526171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ed5d3-7a19-462b-a47e-2b1dd921eb0e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:13.526171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ed5d3-7a19-462b-a47e-2b1dd921eb0e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:13.527505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 591/1162] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:13.532706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99549c-9c4f-4a06-8ade-611efb87f4d7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:13.533098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99549c-9c4f-4a06-8ade-611efb87f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:13.533253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99549c-9c4f-4a06-8ade-611efb87f4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:13.550856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4b99549c-9c4f-4a06-8ade-611efb87f4d7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:13.552043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4b99549c-9c4f-4a06-8ade-611efb87f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:13.574422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:13.617651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 572/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:13.644748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ed5d3-7a19-462b-a47e-2b1dd921eb0e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:13.745188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ed5d3-7a19-462b-a47e-2b1dd921eb0e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:13.778416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:13.778941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:13.779221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:13.779486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:13.808500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:13.847224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:13.848646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:13.945595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 592/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:13.951795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99549c-9c4f-4a06-8ade-611efb87f4d7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:14.047548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b99549c-9c4f-4a06-8ade-611efb87f4d7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:14.080272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c3e82-faf7-4f41-ba7d-36ef95f7132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:14.080887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3e82-faf7-4f41-ba7d-36ef95f7132a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:14.081169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3e82-faf7-4f41-ba7d-36ef95f7132a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:14.081440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c3e82-faf7-4f41-ba7d-36ef95f7132a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:14.087853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c3e82-faf7-4f41-ba7d-36ef95f7132a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:14.089292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c3e82-faf7-4f41-ba7d-36ef95f7132a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-768c3e82-faf7-4f41-ba7d-36ef95f7132a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:14.100940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:14.102134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:16:14.103186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 573/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:16:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:14.109031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:14.167627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:14.205744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a99b7-3b58-4eb4-b1c1-b916e2f19740 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:14.239132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6eeeaa-59d7-429d-aa46-24d5f7b97c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:14.239717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eeeaa-59d7-429d-aa46-24d5f7b97c4c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:14.239995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eeeaa-59d7-429d-aa46-24d5f7b97c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:14.240267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6eeeaa-59d7-429d-aa46-24d5f7b97c4c None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:14.240894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6eeeaa-59d7-429d-aa46-24d5f7b97c4c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:14.241194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6eeeaa-59d7-429d-aa46-24d5f7b97c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:14.242561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6eeeaa-59d7-429d-aa46-24d5f7b97c4c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:14.249653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 593/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 02:16:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:16.232130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0052cb0-a15f-456e-a7e9-9f33a5031463 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.232660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0052cb0-a15f-456e-a7e9-9f33a5031463 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.232660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0052cb0-a15f-456e-a7e9-9f33a5031463 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.234452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 574/1167] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:16.239757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba1a80-ae96-4451-8513-77c610d38546 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.240159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba1a80-ae96-4451-8513-77c610d38546 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.240300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba1a80-ae96-4451-8513-77c610d38546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.258707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-19ba1a80-ae96-4451-8513-77c610d38546 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:16.259880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-19ba1a80-ae96-4451-8513-77c610d38546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:16.323763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 594/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:16.346260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0052cb0-a15f-456e-a7e9-9f33a5031463 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:16.365879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:16.443981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0052cb0-a15f-456e-a7e9-9f33a5031463 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:16.476238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be8c5f-a0ea-4ea2-9668-09bd9916648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:16.476819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be8c5f-a0ea-4ea2-9668-09bd9916648c None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:16.477098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be8c5f-a0ea-4ea2-9668-09bd9916648c None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:16.477366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be8c5f-a0ea-4ea2-9668-09bd9916648c None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:16.478013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be8c5f-a0ea-4ea2-9668-09bd9916648c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:16.479388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be8c5f-a0ea-4ea2-9668-09bd9916648c None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:16.551325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 575/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Mon Aug 8 02:16:16 2022] POST /identity/v3/domains => generated 267 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:16:18.550776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86399b93-4a1a-44e9-a0ff-292f8895cd64 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:18.551363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86399b93-4a1a-44e9-a0ff-292f8895cd64 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:18.551363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86399b93-4a1a-44e9-a0ff-292f8895cd64 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:18.552647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 595/1170] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:18.557684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812a88d-31be-42a9-a9c8-c477c4b63db6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:18.558112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812a88d-31be-42a9-a9c8-c477c4b63db6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:18.558212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812a88d-31be-42a9-a9c8-c477c4b63db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:18.576218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5812a88d-31be-42a9-a9c8-c477c4b63db6 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:18.578135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5812a88d-31be-42a9-a9c8-c477c4b63db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:18.641754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 576/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:18.664425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86399b93-4a1a-44e9-a0ff-292f8895cd64 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:18.720499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:18.767621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86399b93-4a1a-44e9-a0ff-292f8895cd64 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:18.803258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf00f7-5e5c-4c7c-bd55-9760e9ccdddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:18.803803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf00f7-5e5c-4c7c-bd55-9760e9ccdddf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:18.804078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf00f7-5e5c-4c7c-bd55-9760e9ccdddf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:18.804345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf00f7-5e5c-4c7c-bd55-9760e9ccdddf None admin] PATH_INFO: `/v3/domains/d552ac91b1cb4d778434e058128c0dcd` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:18.814822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf00f7-5e5c-4c7c-bd55-9760e9ccdddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=d552ac91b1cb4d778434e058128c0dcd)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:18.816236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf00f7-5e5c-4c7c-bd55-9760e9ccdddf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:18.817778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 596/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Aug 8 02:16:18 2022] GET /identity/v3/domains/d552ac91b1cb4d778434e058128c0dcd => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:18.823592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812a88d-31be-42a9-a9c8-c477c4b63db6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:18.834092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:18.919372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812a88d-31be-42a9-a9c8-c477c4b63db6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:18.954179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0025c1-75fc-4d8f-bfd1-49373d14ef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:18.954788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0025c1-75fc-4d8f-bfd1-49373d14ef89 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:18.955066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0025c1-75fc-4d8f-bfd1-49373d14ef89 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:18.955343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0025c1-75fc-4d8f-bfd1-49373d14ef89 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:18.955977 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0025c1-75fc-4d8f-bfd1-49373d14ef89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:18.957407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0025c1-75fc-4d8f-bfd1-49373d14ef89 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:19.041664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 577/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:16:18 2022] POST /identity/v3/projects => generated 361 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:16:21.056627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b319-2142-4eda-b2dc-e45fd3d16823 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:21.057128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b319-2142-4eda-b2dc-e45fd3d16823 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:21.057214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6b319-2142-4eda-b2dc-e45fd3d16823 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:21.058635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 597/1174] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:21.064384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063ef0-ccf8-4776-a62a-0b10ce61d8d4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:21.064780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063ef0-ccf8-4776-a62a-0b10ce61d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:21.064920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063ef0-ccf8-4776-a62a-0b10ce61d8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:21.083156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e1063ef0-ccf8-4776-a62a-0b10ce61d8d4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:21.084325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e1063ef0-ccf8-4776-a62a-0b10ce61d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:21.137297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:21.148623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 578/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:21.173124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b319-2142-4eda-b2dc-e45fd3d16823 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:21.273455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6b319-2142-4eda-b2dc-e45fd3d16823 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:21.289224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:21.306677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdea6b-0bd3-4a37-80d1-c14b387b6d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:21.307282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdea6b-0bd3-4a37-80d1-c14b387b6d1f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:21.307561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdea6b-0bd3-4a37-80d1-c14b387b6d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:21.307835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdea6b-0bd3-4a37-80d1-c14b387b6d1f None admin] PATH_INFO: `/v3/domains/d552ac91b1cb4d778434e058128c0dcd` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:21.308969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdea6b-0bd3-4a37-80d1-c14b387b6d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=d552ac91b1cb4d778434e058128c0dcd)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:21.310410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdea6b-0bd3-4a37-80d1-c14b387b6d1f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:21.312558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 598/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Aug 8 02:16:21 2022] GET /identity/v3/domains/d552ac91b1cb4d778434e058128c0dcd => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:21.318262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1063ef0-ccf8-4776-a62a-0b10ce61d8d4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:21.421204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1063ef0-ccf8-4776-a62a-0b10ce61d8d4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:21.454149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170cfe8-111b-4df9-baf4-38789c94fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:21.454707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cfe8-111b-4df9-baf4-38789c94fab0 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:21.454985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cfe8-111b-4df9-baf4-38789c94fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:21.455252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cfe8-111b-4df9-baf4-38789c94fab0 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:21.455882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170cfe8-111b-4df9-baf4-38789c94fab0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:21.457316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170cfe8-111b-4df9-baf4-38789c94fab0 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:21.550404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 579/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:16:21 2022] POST /identity/v3/users => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:16:23.576080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dbff7-2ebe-49d0-8d40-f06890c669db None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.576846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dbff7-2ebe-49d0-8d40-f06890c669db None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.576846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dbff7-2ebe-49d0-8d40-f06890c669db None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.577977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 599/1178] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:23.582992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825c83c-1de1-4dbe-a7d9-e0fd640f54f8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.583367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825c83c-1de1-4dbe-a7d9-e0fd640f54f8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.583535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825c83c-1de1-4dbe-a7d9-e0fd640f54f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.601120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d825c83c-1de1-4dbe-a7d9-e0fd640f54f8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:23.602275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d825c83c-1de1-4dbe-a7d9-e0fd640f54f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:23.643229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:23.666480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 580/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:23.690225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dbff7-2ebe-49d0-8d40-f06890c669db None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:23.789750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dbff7-2ebe-49d0-8d40-f06890c669db None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:23.823914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:23.824515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.824791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.825059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.835284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:23.839782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:23.840932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:16:23.842294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 600/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:23.847444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825c83c-1de1-4dbe-a7d9-e0fd640f54f8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:23.942765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825c83c-1de1-4dbe-a7d9-e0fd640f54f8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:23.975417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:23.976034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:23.976313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:23.976582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:23.977235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:23.977535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:23.978926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:23.987644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 581/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:23.993338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.037919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.088802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea145a-b7a9-48ad-bbbf-db709ec7b304 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.121964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.122592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.122912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.123205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] PATH_INFO: `/v3/users/c620fea1355640ac9ccab0d0dd55e0de` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.135695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] RBAC: Authorizing `identity:get_user(user_id=c620fea1355640ac9ccab0d0dd55e0de)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.137157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.138787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 601/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:16:23 2022] GET /identity/v3/users/c620fea1355640ac9ccab0d0dd55e0de => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:24.144670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.184286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.242599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9efe8-f00a-4c50-a7c9-4da59166173a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.274737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.275325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.275601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.275976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.341313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.343348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.380552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.439033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 582/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:24.444959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.542271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed526e-bfab-407f-92c6-0a956b2b0187 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.543802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:24.576923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88ae17-b82f-4341-86e3-8c5b97603dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.577599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ae17-b82f-4341-86e3-8c5b97603dae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.577795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ae17-b82f-4341-86e3-8c5b97603dae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.578063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88ae17-b82f-4341-86e3-8c5b97603dae None admin] PATH_INFO: `/v3/projects/e2ee4fc86c0e444e952489a34fe915ef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.579212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ae17-b82f-4341-86e3-8c5b97603dae None admin] RBAC: Authorizing `identity:get_project(project_id=e2ee4fc86c0e444e952489a34fe915ef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.580617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88ae17-b82f-4341-86e3-8c5b97603dae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.582210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 602/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/projects/e2ee4fc86c0e444e952489a34fe915ef => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:24.588183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:24.684084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b05568-f5b0-430f-bbeb-2d48434697b4 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:24.716364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71c0df-c58a-4c27-ad72-fed4ebb3fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:24.717643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71c0df-c58a-4c27-ad72-fed4ebb3fdbf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:24.717917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71c0df-c58a-4c27-ad72-fed4ebb3fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:24.718187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71c0df-c58a-4c27-ad72-fed4ebb3fdbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:24.718925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71c0df-c58a-4c27-ad72-fed4ebb3fdbf None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=e2ee4fc86c0e444e952489a34fe915ef, user.id=c620fea1355640ac9ccab0d0dd55e0de) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:24.719231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71c0df-c58a-4c27-ad72-fed4ebb3fdbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:24.720641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71c0df-c58a-4c27-ad72-fed4ebb3fdbf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:24.725774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 583/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:24 2022] GET /identity/v3/role_assignments?user.id=c620fea1355640ac9ccab0d0dd55e0de&scope.project.id=e2ee4fc86c0e444e952489a34fe915ef&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:26.661333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de14c0-40e9-443b-8f40-02fcccf9be04 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:26.661899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de14c0-40e9-443b-8f40-02fcccf9be04 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:26.661899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de14c0-40e9-443b-8f40-02fcccf9be04 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:26.663320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 603/1186] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:26.668297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d239-7aef-43c1-919a-e65c4483b173 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:26.668707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d239-7aef-43c1-919a-e65c4483b173 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:26.668842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d239-7aef-43c1-919a-e65c4483b173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:26.675917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:26.688147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-91b7d239-7aef-43c1-919a-e65c4483b173 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:26.689311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-91b7d239-7aef-43c1-919a-e65c4483b173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:26.754474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 584/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:26.776414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de14c0-40e9-443b-8f40-02fcccf9be04 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:26.876658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de14c0-40e9-443b-8f40-02fcccf9be04 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:26.908837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:26.909437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:26.909710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:26.909977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:26.920694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:26.925216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:26.926315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:16:26.927205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 604/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:26.932274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7d239-7aef-43c1-919a-e65c4483b173 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:27.026400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7d239-7aef-43c1-919a-e65c4483b173 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:27.054938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:27.059731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:27.060352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:27.060623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:27.060895 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:27.061537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:27.061833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:27.063210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:27.070931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 585/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:27.076858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:27.108203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:27.175932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca5955-98d8-42e9-98b5-1476d4be3b5f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:27.208422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:27.209024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:27.209301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:27.209570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] PATH_INFO: `/v3/users/c620fea1355640ac9ccab0d0dd55e0de` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:27.210701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] RBAC: Authorizing `identity:get_user(user_id=c620fea1355640ac9ccab0d0dd55e0de)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:27.212105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:27.213631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 605/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:16:27 2022] GET /identity/v3/users/c620fea1355640ac9ccab0d0dd55e0de => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:27.219951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:27.316543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476477be-f853-44d8-a90a-8a533fb7f98d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:27.349408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042ffef-65cc-4ca9-a53f-3910eb665ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:27.350021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042ffef-65cc-4ca9-a53f-3910eb665ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:27.350297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042ffef-65cc-4ca9-a53f-3910eb665ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:27.350594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042ffef-65cc-4ca9-a53f-3910eb665ff5 None admin] PATH_INFO: `/v3/projects/e2ee4fc86c0e444e952489a34fe915ef` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:27.351696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042ffef-65cc-4ca9-a53f-3910eb665ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=e2ee4fc86c0e444e952489a34fe915ef)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:27.353116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042ffef-65cc-4ca9-a53f-3910eb665ff5 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:27.354763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 586/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:16:27 2022] GET /identity/v3/projects/e2ee4fc86c0e444e952489a34fe915ef => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:27.360675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:27.455203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:27.458810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c95ee3-12b6-486e-9929-e868b9dd690f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:27.492409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de682c4-9dfd-4724-8a82-4a566363292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:27.493041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de682c4-9dfd-4724-8a82-4a566363292c None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:27.493320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de682c4-9dfd-4724-8a82-4a566363292c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:27.493592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de682c4-9dfd-4724-8a82-4a566363292c None admin] PATH_INFO: `/v3/projects/e2ee4fc86c0e444e952489a34fe915ef/users/c620fea1355640ac9ccab0d0dd55e0de/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:27.495486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de682c4-9dfd-4724-8a82-4a566363292c None admin] RBAC: Authorizing `identity:create_grant(project_id=e2ee4fc86c0e444e952489a34fe915ef, user_id=c620fea1355640ac9ccab0d0dd55e0de, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:27.496902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de682c4-9dfd-4724-8a82-4a566363292c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:27.571956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 606/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:16:27 2022] PUT /identity/v3/projects/e2ee4fc86c0e444e952489a34fe915ef/users/c620fea1355640ac9ccab0d0dd55e0de/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:16:29.534122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8b2e8-3caa-43d5-b5f9-5eb5e7393516 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.534708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8b2e8-3caa-43d5-b5f9-5eb5e7393516 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.534708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8b2e8-3caa-43d5-b5f9-5eb5e7393516 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.536036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 587/1193] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:16:29.541098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece67a6-4848-4101-a29d-a632c6263c7d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.541516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece67a6-4848-4101-a29d-a632c6263c7d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.541644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece67a6-4848-4101-a29d-a632c6263c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.561409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5ece67a6-4848-4101-a29d-a632c6263c7d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:16:29.562626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5ece67a6-4848-4101-a29d-a632c6263c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:16:29.627619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 607/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:16:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:16:29.649659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8b2e8-3caa-43d5-b5f9-5eb5e7393516 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:29.711526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:29.750692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8b2e8-3caa-43d5-b5f9-5eb5e7393516 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:29.783594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:29.784238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.784466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.784732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.796260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:16:29.800824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:16:29.801935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:16:29.803046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 588/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:16:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:16:29.807991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece67a6-4848-4101-a29d-a632c6263c7d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:29.891729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:29.903040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece67a6-4848-4101-a29d-a632c6263c7d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:29.937671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:29.938272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:29.938576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:29.938842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:29.939499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:29.939798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:29.941142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:29.949303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 608/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:29.954913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:30.051465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903a780-5267-4c2e-9b1e-07bfcc712a7d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:30.072724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:30.083937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:30.085351 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:30.085351 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:30.085351 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] PATH_INFO: `/v3/users/c620fea1355640ac9ccab0d0dd55e0de` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:30.086280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] RBAC: Authorizing `identity:get_user(user_id=c620fea1355640ac9ccab0d0dd55e0de)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:30.087764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:30.089358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 589/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 02:16:29 2022] GET /identity/v3/users/c620fea1355640ac9ccab0d0dd55e0de => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:30.095578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:30.192631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341f3c-aa28-46fe-ada3-bc24312b489b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:30.224901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:30.225481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:30.225771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:30.226036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:30.244191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:30.292704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:30.294125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:30.389240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 609/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:16:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:16:30.394838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:30.490635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fce072-d5a9-46c7-983f-c7b38e479c4f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:30.522592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e7212-8e41-4cff-9052-d5dee0718f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:30.523199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e7212-8e41-4cff-9052-d5dee0718f6e None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:30.523476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e7212-8e41-4cff-9052-d5dee0718f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:30.523744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e7212-8e41-4cff-9052-d5dee0718f6e None admin] PATH_INFO: `/v3/projects/e2ee4fc86c0e444e952489a34fe915ef` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:30.524837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e7212-8e41-4cff-9052-d5dee0718f6e None admin] RBAC: Authorizing `identity:get_project(project_id=e2ee4fc86c0e444e952489a34fe915ef)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:30.526256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e7212-8e41-4cff-9052-d5dee0718f6e None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:30.528014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 590/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 02:16:30 2022] GET /identity/v3/projects/e2ee4fc86c0e444e952489a34fe915ef => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:30.533965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:16:30.581974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:30.631407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e069e46-412c-4635-b207-98b82a1e7f51 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:16:30.665133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f87f8-e854-4dbe-82e7-53757d7d398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:16:30.665656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f87f8-e854-4dbe-82e7-53757d7d398f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:16:30.665932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f87f8-e854-4dbe-82e7-53757d7d398f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:16:30.666200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f87f8-e854-4dbe-82e7-53757d7d398f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:16:30.666941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f87f8-e854-4dbe-82e7-53757d7d398f None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=e2ee4fc86c0e444e952489a34fe915ef, user.id=c620fea1355640ac9ccab0d0dd55e0de) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:16:30.667246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f87f8-e854-4dbe-82e7-53757d7d398f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:16:30.669226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f87f8-e854-4dbe-82e7-53757d7d398f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:16:30.676452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 610/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:16:30 2022] GET /identity/v3/role_assignments?user.id=c620fea1355640ac9ccab0d0dd55e0de&scope.project.id=e2ee4fc86c0e444e952489a34fe915ef&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:16:40.518547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=71 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:16:40.673928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=43 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:41.574447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f809ab-ce7b-451f-9246-49ca94ade341 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:41.575043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f809ab-ce7b-451f-9246-49ca94ade341 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:41.575043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f809ab-ce7b-451f-9246-49ca94ade341 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:41.576472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 591/1201] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:41.582432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0d382-785e-4dea-9ae8-25c51afcb892 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:41.586148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0d382-785e-4dea-9ae8-25c51afcb892 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:41.586148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0d382-785e-4dea-9ae8-25c51afcb892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:41.600742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6ab0d382-785e-4dea-9ae8-25c51afcb892 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:41.601925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6ab0d382-785e-4dea-9ae8-25c51afcb892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:41.667955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 611/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:17:41.696079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f809ab-ce7b-451f-9246-49ca94ade341 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:41.799564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f809ab-ce7b-451f-9246-49ca94ade341 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:41.833713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:41.834262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:41.834571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:41.834837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:41.844999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:41.846658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:41.860256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:41.866921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:41.868167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:41.868167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:41.868167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:41.868167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 592/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:41.868418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0d382-785e-4dea-9ae8-25c51afcb892 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:41.965521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0d382-785e-4dea-9ae8-25c51afcb892 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:41.996196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:42.002608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7b0d1-38d2-4109-abc1-0247c3634eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:42.003204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7b0d1-38d2-4109-abc1-0247c3634eef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:42.003478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7b0d1-38d2-4109-abc1-0247c3634eef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:42.003746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7b0d1-38d2-4109-abc1-0247c3634eef None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:42.004373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7b0d1-38d2-4109-abc1-0247c3634eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:42.004675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7b0d1-38d2-4109-abc1-0247c3634eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:42.006019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7b0d1-38d2-4109-abc1-0247c3634eef None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:42.016882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 612/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:42.022708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:42.074559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:42.121341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087fb78-8e44-4f1a-a4bb-28e5514ff7ca None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:42.153372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d6da8-db2a-4710-a791-2bd7dfeca440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:42.153962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d6da8-db2a-4710-a791-2bd7dfeca440 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:42.154236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d6da8-db2a-4710-a791-2bd7dfeca440 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:42.154533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d6da8-db2a-4710-a791-2bd7dfeca440 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:42.155152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d6da8-db2a-4710-a791-2bd7dfeca440 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:42.156672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d6da8-db2a-4710-a791-2bd7dfeca440 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:42.302708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 593/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 02:17:42 2022] POST /identity/v3/users => generated 251 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:17:44.311657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b145-bf99-485b-9f2f-c2b2aa94077a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.312187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b145-bf99-485b-9f2f-c2b2aa94077a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.312187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517b145-bf99-485b-9f2f-c2b2aa94077a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.314292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 613/1206] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:44.319225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3720-b93d-4e4a-a75e-4ea1dde77e89 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.321233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3720-b93d-4e4a-a75e-4ea1dde77e89 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.321233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3720-b93d-4e4a-a75e-4ea1dde77e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.339232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-bcde3720-b93d-4e4a-a75e-4ea1dde77e89 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:44.340422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bcde3720-b93d-4e4a-a75e-4ea1dde77e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:44.405551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 594/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:17:44.428390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b145-bf99-485b-9f2f-c2b2aa94077a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:44.532794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517b145-bf99-485b-9f2f-c2b2aa94077a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:44.568765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:44.569303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.569578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.569845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.580695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:17:44.585449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:44.586597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:44.587822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 614/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:17:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:44.592725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde3720-b93d-4e4a-a75e-4ea1dde77e89 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:44.632732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:44.692021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde3720-b93d-4e4a-a75e-4ea1dde77e89 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:44.725053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:44.725649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.725925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.726193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.726873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:44.727174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:44.728540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:44.737074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 595/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:17:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:44.742906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:44.793368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:44.840961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74a3b6-6b1e-492c-af1c-4f74212da181 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:44.877907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:44.878563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.878823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.879090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.888800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:44.890218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:44.904542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:44.910613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:44.911507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:44.911507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:44.911507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:44.911507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 615/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:44.913021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:45.010682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:45.013617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f203be5-519b-4fec-abe2-acac067a37cd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:45.047700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:45.048345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:45.048634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:45.049093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:45.049887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:45.050537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:45.056507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:45.063846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 596/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:45.070023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:45.169981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b66bd-569c-47d0-936c-10e64ab5aa03 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:45.171509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:45.203821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:45.205106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:45.205471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:45.205741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:45.212906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:45.214469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:17:45.225873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:45.227032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:45.228343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 616/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:45.233512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:45.332196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5728c29-79e0-4c39-a33e-283a9990e46d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:45.366295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:45.366930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:45.367211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:45.367489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:45.368210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:45.368515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:45.369937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:45.387252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 597/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 02:17:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:45.393952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:45.494009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8564c77-de0f-4bf7-92f5-a6ae9d0b6037 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:45.527337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:45.527957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:45.528232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:45.528498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:45.558340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:45.596382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:45.597840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:45.697852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 617/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:17:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:17:45.703675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:45.723042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:45.801048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58741f7b-0f92-428a-a84b-19377578756f None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:45.835778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:45.836387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:45.836659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:45.836921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:45.847371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:45.850088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:45.863961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:45.864942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:45.865845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:45.865845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:45.865845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:45.865845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 598/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:45.871040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:45.932661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:45.970305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3095a8-31af-4ef4-af9d-6f38624ae3b2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:46.003554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:46.004140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:46.004417 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:46.004684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:46.005311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:46.005607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:46.006982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:46.018251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 618/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:46.023996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:46.093181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:46.125321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896c12c-b0c0-4b3e-802e-5bef47e0b285 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:46.159134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:46.159743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:46.160016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:46.160279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:46.167444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:46.171358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:17:46.180256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:46.181304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:46.182625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 599/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:46.187409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:46.286820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114b1d7-48f2-4de5-8976-b8a55364f711 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:46.308875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:46.320014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dfb94-fb08-42a8-9f15-f775a1c7d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:46.320604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dfb94-fb08-42a8-9f15-f775a1c7d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:46.320878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dfb94-fb08-42a8-9f15-f775a1c7d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:46.321142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dfb94-fb08-42a8-9f15-f775a1c7d4f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:46.321816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dfb94-fb08-42a8-9f15-f775a1c7d4f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:46.322115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dfb94-fb08-42a8-9f15-f775a1c7d4f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:46.323559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dfb94-fb08-42a8-9f15-f775a1c7d4f2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:46.335596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 619/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:46.341762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:46.440676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff0926-965b-4429-8a67-25c632347dbd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:46.468427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:46.474812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee05f34-323c-4849-8bc6-53c2fce3e7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:46.475426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee05f34-323c-4849-8bc6-53c2fce3e7da None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:46.475685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee05f34-323c-4849-8bc6-53c2fce3e7da None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:46.475955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee05f34-323c-4849-8bc6-53c2fce3e7da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:46.476719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee05f34-323c-4849-8bc6-53c2fce3e7da None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=8547ea522755494ba5177d120561dcff) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:46.477005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee05f34-323c-4849-8bc6-53c2fce3e7da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:46.478442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee05f34-323c-4849-8bc6-53c2fce3e7da None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:46.483796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 600/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:17:46 2022] GET /identity/v3/role_assignments?user.id=8547ea522755494ba5177d120561dcff&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:17:48.461825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36df710-59e1-4fa6-bc34-e6d51053ba9b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:48.462404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36df710-59e1-4fa6-bc34-e6d51053ba9b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:48.462404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36df710-59e1-4fa6-bc34-e6d51053ba9b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:48.463757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 620/1220] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:48.468905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228329-2a86-4984-b1eb-a18b2e6730c3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:48.469299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228329-2a86-4984-b1eb-a18b2e6730c3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:48.469484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228329-2a86-4984-b1eb-a18b2e6730c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:48.488342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4f228329-2a86-4984-b1eb-a18b2e6730c3 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:48.489538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4f228329-2a86-4984-b1eb-a18b2e6730c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:48.555923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 601/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:17:48.578056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36df710-59e1-4fa6-bc34-e6d51053ba9b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:48.677959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36df710-59e1-4fa6-bc34-e6d51053ba9b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:48.715485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:48.716061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:48.716338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:48.716609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:48.726504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:17:48.731175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:48.736587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:48.737641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 621/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:17:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:48.739561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f228329-2a86-4984-b1eb-a18b2e6730c3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:48.835311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f228329-2a86-4984-b1eb-a18b2e6730c3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:48.858832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:48.869655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:48.870245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:48.870550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:48.870820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:48.871541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:48.871838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:48.873190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:48.880881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 602/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:17:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:48.886870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:48.932451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:48.989543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff151fd7-624f-4beb-8bf5-b5648610a9e6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:49.022637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:49.023278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:49.023548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:49.023811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:49.034325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:49.035813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:49.052172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:49.053338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:49.054379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:49.054379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:49.054379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:49.054379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 622/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:49.058985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:49.158590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e2713-38bd-4968-acac-25efdf83fba7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:49.191260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:49.191847 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:49.192122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:49.192392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:49.193013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:49.193308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:49.194691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:49.206839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 603/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:49.212657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:49.305269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:49.311799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485cc92-c837-4782-8646-3a6c77bee399 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:49.345559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:49.346179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:49.346519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:49.346796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:49.353766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:49.357323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:17:49.367354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:49.368429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:49.373875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 623/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:49.374506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:49.420952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:49.472699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e061-9048-4929-93c9-c45f12a2cd39 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:49.506516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:49.507102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:49.507374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:49.507638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:49.508364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:49.508642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:49.510167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:49.527103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 604/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 02:17:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:49.532913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:49.634336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f703226-4ed4-443b-8cd9-763deea71790 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:49.667698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:49.668306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:49.668583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:49.668851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:49.679117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:49.680530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:49.694773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:49.701548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:49.702686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:49.702686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:49.702686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:49.702686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 624/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:49.702975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:49.799514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34898087-cfa1-4ef1-b0e9-5910c8092f3b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:49.801780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:49.833434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:49.834033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:49.834302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:49.834601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:49.835224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:49.835519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:49.836876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:49.847604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 605/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:49.853297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:49.867583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:49.964533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe21ab-8bdb-4dcf-a0a4-a209e67f38b2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:49.998634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:49.999245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:49.999516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:49.999781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:50.007765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:50.009253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:17:50.021168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:50.022061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:50.022992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:50.026493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 625/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:50.028058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:50.127164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84de34-4477-4bff-a706-e032e9e8fc45 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:50.159468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965660d-6e2a-4c51-ae5d-a7f1670457c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:50.160059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965660d-6e2a-4c51-ae5d-a7f1670457c7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:50.160339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965660d-6e2a-4c51-ae5d-a7f1670457c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:50.160609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965660d-6e2a-4c51-ae5d-a7f1670457c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:50.161288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965660d-6e2a-4c51-ae5d-a7f1670457c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:50.161592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965660d-6e2a-4c51-ae5d-a7f1670457c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:50.163044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965660d-6e2a-4c51-ae5d-a7f1670457c7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:50.175371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 606/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:50.182030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:50.247525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:50.281156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b3895-6d05-4f9b-9183-99aed722c7a7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:50.315460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4b017-e884-4ca9-b6b3-c842fcdc6628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:50.316101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4b017-e884-4ca9-b6b3-c842fcdc6628 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:50.316536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4b017-e884-4ca9-b6b3-c842fcdc6628 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:50.316777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4b017-e884-4ca9-b6b3-c842fcdc6628 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/8547ea522755494ba5177d120561dcff/roles/31a3f360a76e4dd9a17451573b41967e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:50.331664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4b017-e884-4ca9-b6b3-c842fcdc6628 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=8547ea522755494ba5177d120561dcff, role_id=31a3f360a76e4dd9a17451573b41967e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:50.333093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4b017-e884-4ca9-b6b3-c842fcdc6628 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:50.397031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 626/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:17:50 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/8547ea522755494ba5177d120561dcff/roles/31a3f360a76e4dd9a17451573b41967e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:17:52.378615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f805-7713-4892-8092-facf18c48b45 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.379139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f805-7713-4892-8092-facf18c48b45 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.379139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736f805-7713-4892-8092-facf18c48b45 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.382374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 607/1233] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:52.386009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e78f-e76b-45a7-b312-8da82aafd5cd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.386452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e78f-e76b-45a7-b312-8da82aafd5cd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.386561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e78f-e76b-45a7-b312-8da82aafd5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.405028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7e43e78f-e76b-45a7-b312-8da82aafd5cd None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:52.406201 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7e43e78f-e76b-45a7-b312-8da82aafd5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:52.472841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 627/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:17:52.496013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736f805-7713-4892-8092-facf18c48b45 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:52.520767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:52.593809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736f805-7713-4892-8092-facf18c48b45 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:52.627563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:52.628174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.628450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.628764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.639416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:17:52.645333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:52.646708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:52.647560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 608/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 02:17:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:52.652542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e78f-e76b-45a7-b312-8da82aafd5cd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:52.699109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:52.750273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e78f-e76b-45a7-b312-8da82aafd5cd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:52.783774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:52.784352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.784629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.784899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.785575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:52.785872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:52.787259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:52.794843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 628/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 02:17:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:52.800750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:52.884348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:52.899814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0bf4d-e419-4972-bc1f-9d43a8946709 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:52.932194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:52.932814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.933092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.933361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.944426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:52.945857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:52.961765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:52.962973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:52.963897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:52.963897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:52.963897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:52.963897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 609/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:52.968668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:53.071620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8cd43-89c2-4c6f-b200-b4c601c2fc7f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:53.080298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:53.104586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:53.105178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.105471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.105754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.106470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:53.106776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:53.108448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:53.120961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 629/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:53.127982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:53.226507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97bf20-4a8e-4868-8c89-13886477b1b7 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:53.259519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:53.260124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.260395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.260660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.263277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:53.268152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:53.269576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:17:53.280147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:53.281291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:53.282472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 610/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:53.287327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:53.387304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9323bfa-8bff-4b4f-bee0-0b00425215f9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:53.420754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:53.421344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.421623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.421894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.423238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:53.423540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:53.424956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:53.442315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 630/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 02:17:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:53.448511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:53.549412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574d3ff-08a6-47e4-aa73-9e7b4151adbd None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:53.581961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:53.582590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.582859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.583129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.643113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:53.649277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:53.650733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:53.746826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 611/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:17:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:17:53.753006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:53.780042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:53.853347 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22feb40-c307-4616-9cf9-f4ae4f648e72 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:53.886915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a835a-b056-4413-a542-391dec12db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:53.887533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a835a-b056-4413-a542-391dec12db9e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:53.887810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a835a-b056-4413-a542-391dec12db9e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:53.888079 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a835a-b056-4413-a542-391dec12db9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:53.898600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a835a-b056-4413-a542-391dec12db9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:53.900023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a835a-b056-4413-a542-391dec12db9e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-949a835a-b056-4413-a542-391dec12db9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:53.914650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:53.915764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:53.916775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:53.916775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:53.916775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:53.916775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 631/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:53.921614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:54.017504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:54.023216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddbd60-1c30-4d62-9060-d28b1a8c31fb None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:54.056182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:54.056770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:54.057043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:54.057315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:54.057958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:54.058255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:54.059703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:54.071141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 612/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:54.077090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a835a-b056-4413-a542-391dec12db9e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:54.157296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:54.178046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a835a-b056-4413-a542-391dec12db9e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:54.212581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:54.213191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:54.213466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:54.213786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:54.221387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:54.222847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:17:54.233963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:54.235155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:54.236202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 632/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:54.241134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:54.342606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c015f01-f1c3-43b7-9680-2b002e072cf2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:54.376155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea24a67-9a3f-48c2-9344-0084bbea7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:54.376708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea24a67-9a3f-48c2-9344-0084bbea7498 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:54.376983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea24a67-9a3f-48c2-9344-0084bbea7498 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:54.377252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea24a67-9a3f-48c2-9344-0084bbea7498 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:54.377928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea24a67-9a3f-48c2-9344-0084bbea7498 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:54.378237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea24a67-9a3f-48c2-9344-0084bbea7498 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:54.379688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea24a67-9a3f-48c2-9344-0084bbea7498 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:54.392507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 613/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:54.398616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:54.496977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc3114-7007-4f6b-8f90-f6ab4a0fd528 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:54.529792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622ba64-b945-468a-b01c-434ad3b33b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:54.530403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622ba64-b945-468a-b01c-434ad3b33b28 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:54.530687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622ba64-b945-468a-b01c-434ad3b33b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:54.530960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622ba64-b945-468a-b01c-434ad3b33b28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:54.531679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622ba64-b945-468a-b01c-434ad3b33b28 None admin] RBAC: Adding query filter params (role.id=31a3f360a76e4dd9a17451573b41967e, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=8547ea522755494ba5177d120561dcff) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:54.532077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622ba64-b945-468a-b01c-434ad3b33b28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:54.533520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622ba64-b945-468a-b01c-434ad3b33b28 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:54.539854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 633/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:17:54 2022] GET /identity/v3/role_assignments?user.id=8547ea522755494ba5177d120561dcff&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=31a3f360a76e4dd9a17451573b41967e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:56.577030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f1c00-f344-4240-a2d7-0b8e7c8571c9 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.577583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f1c00-f344-4240-a2d7-0b8e7c8571c9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.577583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f1c00-f344-4240-a2d7-0b8e7c8571c9 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.579103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 614/1247] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:56.584293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632f8e-6b91-4f72-b5bb-85376d07806d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.584713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632f8e-6b91-4f72-b5bb-85376d07806d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.584842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632f8e-6b91-4f72-b5bb-85376d07806d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.591121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:56.603456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7e632f8e-6b91-4f72-b5bb-85376d07806d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:56.608050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7e632f8e-6b91-4f72-b5bb-85376d07806d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:56.669972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 634/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:17:56.693604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f1c00-f344-4240-a2d7-0b8e7c8571c9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:56.711830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:56.792345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f1c00-f344-4240-a2d7-0b8e7c8571c9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:56.825635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:56.826246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.826562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.826832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.837160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:17:56.841709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:56.842946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:56.843975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 615/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:17:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:56.849718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e632f8e-6b91-4f72-b5bb-85376d07806d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:56.949458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e632f8e-6b91-4f72-b5bb-85376d07806d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:56.978079 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:56.983235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:56.983863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.984137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.984437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.985710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:56.986012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:56.987413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:56.995465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 635/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:17:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:57.001917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:57.077887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:57.100875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000977e-59a2-4f3f-84fd-e4d90be48338 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:57.134211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:57.134853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:57.135131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:57.135401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:57.145717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:57.147206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:57.162008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:57.163191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:57.164388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:57.164388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:57.164388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:57.164388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 616/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:57.169019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:57.271009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6984ce-9f5d-4b35-8b64-0de1bfd8a2fd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:57.304263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:57.304851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:57.305129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:57.305462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:57.306093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:57.306411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:57.310294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:57.319631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 636/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:17:57.325469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:57.426567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9c64b-8ac1-4b54-b2f4-36018369757b None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:57.451523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:57.459617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:57.460232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:57.460506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:57.460771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:57.467818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:57.469936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:17:57.480413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:57.488638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:17:57.489578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 617/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:57.490172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:57.544311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:57.587323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ebbd9-c008-4437-9197-86e03c387abc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:57.622517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:57.623110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:57.623388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:57.623658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:57.624379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:57.624709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:57.626315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:57.643572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 637/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 02:17:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:57.649403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:57.746933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa436-87e0-40e2-ad3c-34eb9e6de648 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:57.779412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:57.780077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:57.780355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:57.780624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:57.835849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:57.847794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:57.849347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:57.945376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 618/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:17:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:17:57.951308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:58.047819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28673d8-9827-49b1-a622-40ed07ba81a2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:58.060432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:58.081441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:58.082056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:58.082332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:58.082634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:58.092243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:58.093662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:17:58.107870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:58.111031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:17:58.112143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:17:58.112143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:17:58.112143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:58.112143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 638/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:58.114847 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:58.201145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:58.212764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1c3b0-b2f8-4bce-a1fb-530a1b23f993 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:58.246508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:58.247102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:58.247377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:58.247639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:58.248256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:58.248551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:58.249991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:58.261517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 619/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:58.267236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:58.370808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3b112-f76f-4785-8f25-655ccb842e57 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:58.403896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:58.404498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:58.404819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:58.405119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:58.411807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:58.413229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:17:58.425800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:17:58.426934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:17:58.427866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 639/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:17:58.432937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:58.533254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c73392-1515-4f64-bc52-9ed824369d78 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:58.568766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4497f-922d-4809-9e12-4c8fff3d5b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:58.569361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4497f-922d-4809-9e12-4c8fff3d5b72 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:58.569641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4497f-922d-4809-9e12-4c8fff3d5b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:58.569913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4497f-922d-4809-9e12-4c8fff3d5b72 None admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:58.570628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4497f-922d-4809-9e12-4c8fff3d5b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:58.570932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4497f-922d-4809-9e12-4c8fff3d5b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:58.572353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4497f-922d-4809-9e12-4c8fff3d5b72 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:58.584596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 620/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:17:58.590908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:58.617925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:58.692789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c244df-5295-4456-a5e4-b94369fb2393 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:58.727945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29d177-3093-4f33-a7fa-e268368dcdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:58.728524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d177-3093-4f33-a7fa-e268368dcdcf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:58.728793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d177-3093-4f33-a7fa-e268368dcdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:58.729054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d177-3093-4f33-a7fa-e268368dcdcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:58.729763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29d177-3093-4f33-a7fa-e268368dcdcf None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=8547ea522755494ba5177d120561dcff) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:17:58.730094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29d177-3093-4f33-a7fa-e268368dcdcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:58.731520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29d177-3093-4f33-a7fa-e268368dcdcf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:58.738056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 640/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:17:58 2022] GET /identity/v3/role_assignments?user.id=8547ea522755494ba5177d120561dcff&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=1b9fb565648c49388166fe130739891f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:00.669327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29775a-9b35-4edf-bf74-29dcf1b2c094 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:00.669949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29775a-9b35-4edf-bf74-29dcf1b2c094 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:00.669949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29775a-9b35-4edf-bf74-29dcf1b2c094 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:00.671224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 621/1261] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:00.676806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e3707-7a8d-47f8-b76e-cdbf33149d03 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:00.677245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e3707-7a8d-47f8-b76e-cdbf33149d03 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:00.677355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e3707-7a8d-47f8-b76e-cdbf33149d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:00.695841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3d0e3707-7a8d-47f8-b76e-cdbf33149d03 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:00.697018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3d0e3707-7a8d-47f8-b76e-cdbf33149d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:00.763224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 641/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:18:00.785405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29775a-9b35-4edf-bf74-29dcf1b2c094 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:00.796308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:00.883778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29775a-9b35-4edf-bf74-29dcf1b2c094 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:00.917984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:00.918642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:00.918923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:00.919193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:00.929939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:18:00.934656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:00.935795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:18:00.936826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 622/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:00.941770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e3707-7a8d-47f8-b76e-cdbf33149d03 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:00.960570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:01.040107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e3707-7a8d-47f8-b76e-cdbf33149d03 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:01.073229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:01.073806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:01.074082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:01.074367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:01.075023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:01.075321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:01.076667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:01.084787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 642/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:01.090752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:01.155690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:01.188092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eef4a8-b1d2-4dc0-ac75-aef41b227b9a None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:01.222325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:01.222986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:01.223258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:01.223519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:01.233909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:01.235737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:18:01.250855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:01.252021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:18:01.253277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:18:01.253277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:01.253277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:18:01.253277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 623/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:01.257698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:01.339465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:01.358713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c3d95-5aaf-45c1-83a8-f22b0f7167a3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:01.392033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:01.392621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:01.392897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:01.393163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:01.393793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:01.394154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:01.395534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:01.407302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 643/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:01.412962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:01.510772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664a86f-8a36-40f8-8411-d7b5262718b3 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:01.531135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:01.545392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:01.546009 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:01.546286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:01.546637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:01.553896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:01.555361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:18:01.566964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:01.568099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:18:01.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 624/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:18:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:01.573977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:01.674437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9182704-4fe0-4090-95f6-3bb707aa8a53 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:01.708681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:01.709304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:01.709543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:01.709814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:01.710584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:01.710895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:01.712308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:01.721900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:01.729733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 644/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 02:18:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:01.735789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:01.835987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c09a7-198e-41b7-9b46-554be1449dcf None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:01.869587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:01.870197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:01.870508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:01.870782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:01.881330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:01.882807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:18:01.896793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:01.897885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:18:01.899149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:18:01.899149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:01.899149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:18:01.899149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 625/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:01.903739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:02.007346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c091962-6a71-4376-b633-7920dce4fe0b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:02.039957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:02.040540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:02.040812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:02.041076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:02.041697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:02.041990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:02.043371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:02.057505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 645/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:02.063899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:02.095404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:02.162248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6c921-3aec-445c-bf1c-a2778ecb6fd6 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:02.195961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:02.196577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:02.196855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:02.197128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:02.203867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:02.205319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:18:02.216416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:02.219938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:18:02.220828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 626/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:02.223863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:02.283252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:02.326006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da32a2-13a8-4320-be7e-b139b6f7278f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:02.360312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aadd51-86cc-4adb-9b3a-7f3bf8ed88c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:02.361021 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadd51-86cc-4adb-9b3a-7f3bf8ed88c2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:02.361568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadd51-86cc-4adb-9b3a-7f3bf8ed88c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:02.362080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aadd51-86cc-4adb-9b3a-7f3bf8ed88c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:02.363270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aadd51-86cc-4adb-9b3a-7f3bf8ed88c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:02.363580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aadd51-86cc-4adb-9b3a-7f3bf8ed88c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:02.365020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aadd51-86cc-4adb-9b3a-7f3bf8ed88c2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:02.377077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 646/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:02.383288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:02.475276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:02.484006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224f755-7dea-4099-8ced-c60b5e5d6b1d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:02.516443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73690ca8-7d98-4236-9599-6ddba7590419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:02.517078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73690ca8-7d98-4236-9599-6ddba7590419 None admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:02.517353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73690ca8-7d98-4236-9599-6ddba7590419 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:02.517622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73690ca8-7d98-4236-9599-6ddba7590419 None admin] PATH_INFO: `/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/8547ea522755494ba5177d120561dcff/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:02.519716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73690ca8-7d98-4236-9599-6ddba7590419 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e3fa8dc6e745569f8ffb7b76448278, user_id=8547ea522755494ba5177d120561dcff, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:02.521189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73690ca8-7d98-4236-9599-6ddba7590419 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:02.620379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 627/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 02:18:02 2022] PUT /identity/v3/projects/94e3fa8dc6e745569f8ffb7b76448278/users/8547ea522755494ba5177d120561dcff/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:18:04.622909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719458-73ac-4ceb-9334-d3b1468d2a4a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:04.623491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719458-73ac-4ceb-9334-d3b1468d2a4a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:04.623491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719458-73ac-4ceb-9334-d3b1468d2a4a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:04.624840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 647/1274] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:18:04.630529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f74219-1f86-4f92-90fd-d801b172eb08 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:04.630965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f74219-1f86-4f92-90fd-d801b172eb08 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:04.631088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f74219-1f86-4f92-90fd-d801b172eb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:04.649734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d1f74219-1f86-4f92-90fd-d801b172eb08 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:04.650973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d1f74219-1f86-4f92-90fd-d801b172eb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:04.717667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 628/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:18:04.741196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12719458-73ac-4ceb-9334-d3b1468d2a4a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:04.845068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12719458-73ac-4ceb-9334-d3b1468d2a4a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:04.862278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:04.881563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:04.881563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:04.881563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:04.881563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:04.889937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:18:04.895306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:04.896458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:04.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 648/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 02:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:04.902460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f74219-1f86-4f92-90fd-d801b172eb08 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:04.983654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:05.002520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f74219-1f86-4f92-90fd-d801b172eb08 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:05.034088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:05.035823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.035823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.035823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.036083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:05.036169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:05.037507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:05.045131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 629/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 02:18:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:05.051591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:05.150574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3711c-8476-45cf-ae43-97e207b9aaef None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:05.184155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:05.184841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.185112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.185386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.195940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:05.197864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:18:05.211733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:05.212923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:18:05.214016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:18:05.214016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:05.214016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:05.214016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 649/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:05.218586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:05.316776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5699d6-8fb7-4caf-b5b9-0dfd616a0c32 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:05.350188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:05.350820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.351095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.351363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.351990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:05.352284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:05.353704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:05.366063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 630/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:05.371780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:05.402548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:05.473011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b9901-93e8-4c34-9d4a-5d3d7ca06e62 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:05.507565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:05.508172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.508443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.508707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.515459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:05.516884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:18:05.527900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:05.529072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:05.530447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 650/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:18:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:05.535291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:05.539243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:05.632911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e7118-821e-4d3d-acd0-556de2d546c2 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:05.665194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:05.665724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.665996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.666264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.666995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:05.667292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:05.668694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:05.685403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 631/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 02:18:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:05.691576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:05.780961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:05.791402 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e76ef2-5c7d-4527-8a7b-ac7c4e09be68 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:05.823575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:05.824159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:05.824515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:05.824757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:05.888643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:05.890064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:05.982673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:05.992667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 651/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 02:18:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:18:05.998455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:06.060200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:06.097586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154825a-25c3-4c3a-906a-164dc09c8b83 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:06.132051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:06.132662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.132940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.133207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.143681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:06.145202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:18:06.159372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:06.163402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:18:06.164502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:18:06.164502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:18:06.164502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:18:06.164502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 632/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:06.166167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:06.266333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63803956-aed6-4a59-ab3c-3435034b6115 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:06.299593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:06.300173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.300444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.300707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.301330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:06.301625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:06.303000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:06.314190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 652/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 02:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:06.319881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:06.416302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ac2a8-6aec-4a0b-b274-cb2161765856 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:06.430270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:06.450078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:06.450727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.451025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.451381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.458748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:06.460182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:18:06.471587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:06.475148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: ERROR keystone.server.flask.application Aug 08 02:18:06.476290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 633/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 02:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:06.478666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:06.529101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:06.577684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf4121-ff81-41d7-9b2c-3f2663573384 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:06.611772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae523e-1ce1-4ba7-9aa7-339a58f98f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:06.612350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae523e-1ce1-4ba7-9aa7-339a58f98f68 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.612622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae523e-1ce1-4ba7-9aa7-339a58f98f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.612889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae523e-1ce1-4ba7-9aa7-339a58f98f68 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.613565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae523e-1ce1-4ba7-9aa7-339a58f98f68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:06.613866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae523e-1ce1-4ba7-9aa7-339a58f98f68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:06.615302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae523e-1ce1-4ba7-9aa7-339a58f98f68 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:06.627328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 653/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 02:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:06.633382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:06.732498 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb17159-fe2d-4a03-b48a-43f1bf156b8d None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:06.767093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e613c-6e00-414c-b33a-e750588bd08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:06.767706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e613c-6e00-414c-b33a-e750588bd08b None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.767980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e613c-6e00-414c-b33a-e750588bd08b None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.768245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e613c-6e00-414c-b33a-e750588bd08b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.768950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e613c-6e00-414c-b33a-e750588bd08b None admin] RBAC: Adding query filter params (role.id=1b9fb565648c49388166fe130739891f, scope.project.id=94e3fa8dc6e745569f8ffb7b76448278, user.id=8547ea522755494ba5177d120561dcff) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:06.769249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e613c-6e00-414c-b33a-e750588bd08b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:06.770690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e613c-6e00-414c-b33a-e750588bd08b None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:06.776190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 634/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 02:18:06 2022] GET /identity/v3/role_assignments?user.id=8547ea522755494ba5177d120561dcff&scope.project.id=94e3fa8dc6e745569f8ffb7b76448278&role.id=1b9fb565648c49388166fe130739891f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:08.934150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490b666-fe91-403e-9c82-4f7c165e3653 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:08.934685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490b666-fe91-403e-9c82-4f7c165e3653 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:08.935146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490b666-fe91-403e-9c82-4f7c165e3653 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:08.936564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 654/1288] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:08.942283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675dc0c-f54d-45dd-a02e-8ce3263e985c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:08.942766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675dc0c-f54d-45dd-a02e-8ce3263e985c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:08.942854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675dc0c-f54d-45dd-a02e-8ce3263e985c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:08.962776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1675dc0c-f54d-45dd-a02e-8ce3263e985c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:08.965564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1675dc0c-f54d-45dd-a02e-8ce3263e985c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:09.000850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:09.045130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 635/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:18:09.074484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490b666-fe91-403e-9c82-4f7c165e3653 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:09.175057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490b666-fe91-403e-9c82-4f7c165e3653 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:09.186174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:09.209885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:09.210527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:09.210801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:09.211071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:09.211640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:09.212990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:09.221419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:09.222533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:09.223463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 655/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:18:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:09.228856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675dc0c-f54d-45dd-a02e-8ce3263e985c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:09.327002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675dc0c-f54d-45dd-a02e-8ce3263e985c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:09.364699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dc8d7-798c-452c-acb0-6677a6d49c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:09.365396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc8d7-798c-452c-acb0-6677a6d49c39 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:09.365673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc8d7-798c-452c-acb0-6677a6d49c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:09.365942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc8d7-798c-452c-acb0-6677a6d49c39 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:09.366595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dc8d7-798c-452c-acb0-6677a6d49c39 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:09.366897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dc8d7-798c-452c-acb0-6677a6d49c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:09.368303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dc8d7-798c-452c-acb0-6677a6d49c39 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:09.373060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 636/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 02:18:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:09.379702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:09.479836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459acbd1-03ad-41b5-86c4-e9175bf6cb89 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:09.513481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f1860-f3a8-4654-b156-4661052a05e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:09.514096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f1860-f3a8-4654-b156-4661052a05e3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:09.514394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f1860-f3a8-4654-b156-4661052a05e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:09.514671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f1860-f3a8-4654-b156-4661052a05e3 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:09.515331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f1860-f3a8-4654-b156-4661052a05e3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:09.515636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f1860-f3a8-4654-b156-4661052a05e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:09.516977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f1860-f3a8-4654-b156-4661052a05e3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:09.521276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 656/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 02:18:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:11.519770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d9029-ecee-40d4-afc8-854908db747e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:11.520304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d9029-ecee-40d4-afc8-854908db747e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.520304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d9029-ecee-40d4-afc8-854908db747e None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.521734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 637/1293] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:11.526969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c677143-afa4-4380-aef3-2e1331265fb7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:11.535015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c677143-afa4-4380-aef3-2e1331265fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.535015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c677143-afa4-4380-aef3-2e1331265fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.545598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6c677143-afa4-4380-aef3-2e1331265fb7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:11.547406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6c677143-afa4-4380-aef3-2e1331265fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:11.576032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:11.613053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 657/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:18:11.635167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d9029-ecee-40d4-afc8-854908db747e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:11.666328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:11.734935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d9029-ecee-40d4-afc8-854908db747e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:11.769997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc6a17-20ff-48c2-bc64-dcca0e5785c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:11.770553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc6a17-20ff-48c2-bc64-dcca0e5785c8 None admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:11.770830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc6a17-20ff-48c2-bc64-dcca0e5785c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.771097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc6a17-20ff-48c2-bc64-dcca0e5785c8 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.771733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc6a17-20ff-48c2-bc64-dcca0e5785c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:11.773088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc6a17-20ff-48c2-bc64-dcca0e5785c8 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:11.842949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 638/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 02:18:11 2022] POST /identity/v3/services => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:18:13.858663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6ded4-200f-4d73-9b8d-c298ed2f1b21 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:13.859273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6ded4-200f-4d73-9b8d-c298ed2f1b21 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:13.859273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6ded4-200f-4d73-9b8d-c298ed2f1b21 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:13.860727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 658/1296] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:13.865930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb06f1-e0e2-4d0e-8fd7-355ceeaaa3d0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:13.866345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb06f1-e0e2-4d0e-8fd7-355ceeaaa3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:13.866507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb06f1-e0e2-4d0e-8fd7-355ceeaaa3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:13.884617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-69bb06f1-e0e2-4d0e-8fd7-355ceeaaa3d0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:13.885789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-69bb06f1-e0e2-4d0e-8fd7-355ceeaaa3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:13.956654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 639/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:18:13.979628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6ded4-200f-4d73-9b8d-c298ed2f1b21 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:14.086071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6ded4-200f-4d73-9b8d-c298ed2f1b21 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:14.119940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:14.120483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:14.120754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:14.121017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:14.121586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:14.122951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:14.132331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:14.133485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:14.134315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 659/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:18:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:14.139665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb06f1-e0e2-4d0e-8fd7-355ceeaaa3d0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:14.157362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:14.233705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb06f1-e0e2-4d0e-8fd7-355ceeaaa3d0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:14.267386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:14.267976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:14.268249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:14.268516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:14.269126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:14.269417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:14.270818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:14.275735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 640/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 02:18:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:14.282317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:14.315056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:14.382719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324c2e8-385b-4b95-b560-a0ac6e0f72b4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:14.420217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1d698-b558-48fc-8470-c86b248dfe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:14.420772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d698-b558-48fc-8470-c86b248dfe43 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:14.421047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d698-b558-48fc-8470-c86b248dfe43 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:14.421312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d698-b558-48fc-8470-c86b248dfe43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:14.422968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1d698-b558-48fc-8470-c86b248dfe43 None admin] RBAC: Adding query filter params (interface=public, service_id=019bc22277d840d1b46354fad5f2ab14, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:14.423268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1d698-b558-48fc-8470-c86b248dfe43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:14.424614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1d698-b558-48fc-8470-c86b248dfe43 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:14.429476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 660/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 02:18:14 2022] GET /identity/v3/endpoints?service_id=019bc22277d840d1b46354fad5f2ab14&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:14.435295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:14.515657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:14.533585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11104abd-cf41-4def-a4c1-8a38d17e7964 None admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:14.566183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e43078-698b-4d8f-b57e-e792f8afd5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:14.566797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e43078-698b-4d8f-b57e-e792f8afd5dd None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:14.567071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e43078-698b-4d8f-b57e-e792f8afd5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:14.567333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e43078-698b-4d8f-b57e-e792f8afd5dd None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:14.567916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e43078-698b-4d8f-b57e-e792f8afd5dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:14.569266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e43078-698b-4d8f-b57e-e792f8afd5dd None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:14.575831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 641/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 02:18:14 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:18:16.567183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39034a-133c-43e4-a043-8d33c59c4b3e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.567799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39034a-133c-43e4-a043-8d33c59c4b3e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.567799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39034a-133c-43e4-a043-8d33c59c4b3e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.569031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 661/1302] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:16.574101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f40a82-e902-44fe-b46d-86e61d0fe0c1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.574540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f40a82-e902-44fe-b46d-86e61d0fe0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.574676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f40a82-e902-44fe-b46d-86e61d0fe0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.592905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-29f40a82-e902-44fe-b46d-86e61d0fe0c1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:16.594159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-29f40a82-e902-44fe-b46d-86e61d0fe0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:16.658587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 642/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:18:16.680617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39034a-133c-43e4-a043-8d33c59c4b3e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:16.774461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.781331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39034a-133c-43e4-a043-8d33c59c4b3e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:16.814677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:16.815298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.815614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.815854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.816435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:16.817832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:18:16.826343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:18:16.828751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Aug 08 02:18:16.829960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 662/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 02:18:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:18:16.834127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f40a82-e902-44fe-b46d-86e61d0fe0c1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:16.909312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.932560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f40a82-e902-44fe-b46d-86e61d0fe0c1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:16.965789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7a8d2-06d9-4e13-9547-0fec76a168cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:16.966404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a8d2-06d9-4e13-9547-0fec76a168cf None admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.966681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a8d2-06d9-4e13-9547-0fec76a168cf None admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.966947 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a8d2-06d9-4e13-9547-0fec76a168cf None admin] PATH_INFO: `/v3/services` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.967554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a8d2-06d9-4e13-9547-0fec76a168cf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:18:16.967846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a8d2-06d9-4e13-9547-0fec76a168cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:16.969362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a8d2-06d9-4e13-9547-0fec76a168cf None admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:16.974286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 643/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 02:18:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:18:16.980192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:17.079330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad287cf-7587-4269-ac16-062a2ff4459a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:17.112291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a2986-4f6f-4905-a6d4-770da06f4efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:17.112872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a2986-4f6f-4905-a6d4-770da06f4efa None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:17.113148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a2986-4f6f-4905-a6d4-770da06f4efa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:17.113416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a2986-4f6f-4905-a6d4-770da06f4efa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:17.114075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a2986-4f6f-4905-a6d4-770da06f4efa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:17.115454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a2986-4f6f-4905-a6d4-770da06f4efa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:17.197803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 663/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:18:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:18:26.971328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=30 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:27.193316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:23.685490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc1f86-1322-497f-872d-5fa546f6bb44 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:23.686080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc1f86-1322-497f-872d-5fa546f6bb44 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:23.686080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc1f86-1322-497f-872d-5fa546f6bb44 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:23.692830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e0d84-0848-43f1-86a3-5cecfe43ba06 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:23.693292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e0d84-0848-43f1-86a3-5cecfe43ba06 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:23.693405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e0d84-0848-43f1-86a3-5cecfe43ba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:23.697799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 644/1307] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:23.823791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-725e0d84-0848-43f1-86a3-5cecfe43ba06 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:23.825001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-725e0d84-0848-43f1-86a3-5cecfe43ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:23.958732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 664/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:24.033661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2377ce-be99-41ce-85e0-500ec5d428b4 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:24.034063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2377ce-be99-41ce-85e0-500ec5d428b4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:24.036149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2377ce-be99-41ce-85e0-500ec5d428b4 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:24.036972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 645/1309] 10.222.0.15 () {54 vars in 965 bytes} [Mon Aug 8 02:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:24.053043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d673-8e2e-4f64-ac46-e1a3a295b344 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:24.053328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d673-8e2e-4f64-ac46-e1a3a295b344 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:24.053599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d673-8e2e-4f64-ac46-e1a3a295b344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:24.166302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e1a7d673-8e2e-4f64-ac46-e1a3a295b344 None None] MFA Rules not processed for user `e8839af749e44fac8a7a311b61463c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:24.167550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e1a7d673-8e2e-4f64-ac46-e1a3a295b344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:24.190094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:24.284942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 665/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Aug 8 02:24:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:24.293808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2377ce-be99-41ce-85e0-500ec5d428b4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:24.295804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fe2377ce-be99-41ce-85e0-500ec5d428b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:24.436488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2377ce-be99-41ce-85e0-500ec5d428b4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:24.516333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7453e47-98e7-4abf-b4fa-a6c2aed085b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:24.517048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453e47-98e7-4abf-b4fa-a6c2aed085b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:24.517327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453e47-98e7-4abf-b4fa-a6c2aed085b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:24.517596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453e47-98e7-4abf-b4fa-a6c2aed085b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:24.520508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f7453e47-98e7-4abf-b4fa-a6c2aed085b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:24.601744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:24.679826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7453e47-98e7-4abf-b4fa-a6c2aed085b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:24.681344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7453e47-98e7-4abf-b4fa-a6c2aed085b9 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:24.706579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 646/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:27.048083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303cf52-851c-4ad9-8a02-df74117be8ed None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:27.048630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303cf52-851c-4ad9-8a02-df74117be8ed None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:27.048809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303cf52-851c-4ad9-8a02-df74117be8ed None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:27.050826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 666/1312] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:27.056122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1852b6-f3ea-42a3-86f4-05f7cc1a5b40 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:27.056534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1852b6-f3ea-42a3-86f4-05f7cc1a5b40 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:27.056670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1852b6-f3ea-42a3-86f4-05f7cc1a5b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:27.128967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5d1852b6-f3ea-42a3-86f4-05f7cc1a5b40 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:27.130148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5d1852b6-f3ea-42a3-86f4-05f7cc1a5b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:27.231125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 647/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:27.269920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303cf52-851c-4ad9-8a02-df74117be8ed None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:27.271831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2303cf52-851c-4ad9-8a02-df74117be8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:27.389766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303cf52-851c-4ad9-8a02-df74117be8ed None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:27.465341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:27.467753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156f276-faee-4b79-b2a4-84acaa232156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:27.468414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156f276-faee-4b79-b2a4-84acaa232156 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:27.468689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156f276-faee-4b79-b2a4-84acaa232156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:27.468954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156f276-faee-4b79-b2a4-84acaa232156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:27.470717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0156f276-faee-4b79-b2a4-84acaa232156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:27.588437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156f276-faee-4b79-b2a4-84acaa232156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:27.589889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156f276-faee-4b79-b2a4-84acaa232156 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:27.614464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 667/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:30.194062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc8382-d87d-4a1e-b772-245511eec0d2 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:30.194598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc8382-d87d-4a1e-b772-245511eec0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:30.194598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc8382-d87d-4a1e-b772-245511eec0d2 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:30.195905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 648/1315] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:30.201078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78edeb-946d-4398-afaa-21834187144a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:30.201501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78edeb-946d-4398-afaa-21834187144a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:30.201614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78edeb-946d-4398-afaa-21834187144a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:30.265597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cd78edeb-946d-4398-afaa-21834187144a None None] MFA Rules not processed for user `82f38e27cd95460f986105f4aeedb6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:30.266838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cd78edeb-946d-4398-afaa-21834187144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:30.317900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:30.361865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 668/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:24:30.401114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc8382-d87d-4a1e-b772-245511eec0d2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:30.402693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-72fc8382-d87d-4a1e-b772-245511eec0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:30.413807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:30.508435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc8382-d87d-4a1e-b772-245511eec0d2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:30.579470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf132ae-e276-4053-8df3-fd11c36cca6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:30.580067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf132ae-e276-4053-8df3-fd11c36cca6d service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:30.580858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf132ae-e276-4053-8df3-fd11c36cca6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:30.581130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf132ae-e276-4053-8df3-fd11c36cca6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:30.582765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6cf132ae-e276-4053-8df3-fd11c36cca6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:30.643546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:30.689151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf132ae-e276-4053-8df3-fd11c36cca6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:30.691393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf132ae-e276-4053-8df3-fd11c36cca6d service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:30.709467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 649/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:33.527746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ef3c6-c49c-4fd6-bb93-29e90d247d1f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:33.528326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ef3c6-c49c-4fd6-bb93-29e90d247d1f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:33.528326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ef3c6-c49c-4fd6-bb93-29e90d247d1f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:33.529609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 669/1318] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:33.534619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babed70-a543-4db0-8ee7-94673cd25067 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:33.535024 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babed70-a543-4db0-8ee7-94673cd25067 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:33.535175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babed70-a543-4db0-8ee7-94673cd25067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:33.599427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9babed70-a543-4db0-8ee7-94673cd25067 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:33.601228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9babed70-a543-4db0-8ee7-94673cd25067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:33.696676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 650/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:33.735664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ef3c6-c49c-4fd6-bb93-29e90d247d1f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:33.737840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5a4ef3c6-c49c-4fd6-bb93-29e90d247d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:33.839091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ef3c6-c49c-4fd6-bb93-29e90d247d1f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:33.910604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf6d3b-ac22-49b8-b8c4-df43ce408849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:33.911198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf6d3b-ac22-49b8-b8c4-df43ce408849 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:33.911471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf6d3b-ac22-49b8-b8c4-df43ce408849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:33.911741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf6d3b-ac22-49b8-b8c4-df43ce408849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:33.913327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b5bf6d3b-ac22-49b8-b8c4-df43ce408849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:33.922489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:34.022333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf6d3b-ac22-49b8-b8c4-df43ce408849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:34.023782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf6d3b-ac22-49b8-b8c4-df43ce408849 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:34.041742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 670/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:37.122725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57fbc-1323-4fbd-bc02-509efc7d8684 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:37.123274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57fbc-1323-4fbd-bc02-509efc7d8684 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:37.123274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57fbc-1323-4fbd-bc02-509efc7d8684 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:37.124632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 651/1321] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:37.130778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a27f3-4fc2-4358-83d5-ea3fa5105fd5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:37.131188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a27f3-4fc2-4358-83d5-ea3fa5105fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:37.131447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a27f3-4fc2-4358-83d5-ea3fa5105fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:37.199359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-597a27f3-4fc2-4358-83d5-ea3fa5105fd5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:37.200467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-597a27f3-4fc2-4358-83d5-ea3fa5105fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:37.261725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:37.301318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 671/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:37.352118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b29aa-9057-46bd-9fc5-4d261899ab8c None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:37.352400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b29aa-9057-46bd-9fc5-4d261899ab8c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:37.352674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b29aa-9057-46bd-9fc5-4d261899ab8c None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:37.354036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 652/1323] 10.222.0.15 () {54 vars in 965 bytes} [Mon Aug 8 02:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:37.361029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730fe6c-1eac-4b8f-a727-307fbc44c627 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:37.361314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730fe6c-1eac-4b8f-a727-307fbc44c627 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:37.361591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730fe6c-1eac-4b8f-a727-307fbc44c627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:37.448595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3730fe6c-1eac-4b8f-a727-307fbc44c627 None None] MFA Rules not processed for user `e8839af749e44fac8a7a311b61463c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:37.449910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3730fe6c-1eac-4b8f-a727-307fbc44c627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:37.529395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 672/1324] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Aug 8 02:24:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:37.539305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b29aa-9057-46bd-9fc5-4d261899ab8c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:37.540819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-af4b29aa-9057-46bd-9fc5-4d261899ab8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:37.560557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:37.645357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b29aa-9057-46bd-9fc5-4d261899ab8c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:37.717630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418b5c5-719f-4723-a2b7-1f8e1b482b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:37.717949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418b5c5-719f-4723-a2b7-1f8e1b482b4f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:37.718236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418b5c5-719f-4723-a2b7-1f8e1b482b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:37.718534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418b5c5-719f-4723-a2b7-1f8e1b482b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:37.720115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8418b5c5-719f-4723-a2b7-1f8e1b482b4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:37.792525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:37.830149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418b5c5-719f-4723-a2b7-1f8e1b482b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:37.834113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418b5c5-719f-4723-a2b7-1f8e1b482b4f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:37.850839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 653/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:40.128664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dbfda-25bc-4c73-88d3-f23959bd0a6a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:40.129182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dbfda-25bc-4c73-88d3-f23959bd0a6a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:40.129301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dbfda-25bc-4c73-88d3-f23959bd0a6a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:40.130740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 673/1326] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:40.135678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a03295-b454-48b6-a177-f603dcd2fc34 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:40.136059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a03295-b454-48b6-a177-f603dcd2fc34 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:40.136238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a03295-b454-48b6-a177-f603dcd2fc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:40.209168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-64a03295-b454-48b6-a177-f603dcd2fc34 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:40.210313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-64a03295-b454-48b6-a177-f603dcd2fc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:40.308062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 654/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:40.344539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dbfda-25bc-4c73-88d3-f23959bd0a6a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:40.346063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f50dbfda-25bc-4c73-88d3-f23959bd0a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:40.416396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:40.461261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dbfda-25bc-4c73-88d3-f23959bd0a6a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:40.535372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa6080-4b16-4fa1-9287-2e6594895b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:40.535965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa6080-4b16-4fa1-9287-2e6594895b47 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:40.536239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa6080-4b16-4fa1-9287-2e6594895b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:40.536503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa6080-4b16-4fa1-9287-2e6594895b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:40.538077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-36aa6080-4b16-4fa1-9287-2e6594895b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:40.644864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:40.647400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa6080-4b16-4fa1-9287-2e6594895b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:40.648841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa6080-4b16-4fa1-9287-2e6594895b47 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:40.666980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 674/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:24:42.733598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec32ed-f3e5-409d-83bf-1bc973fab328 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:42.734124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec32ed-f3e5-409d-83bf-1bc973fab328 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:42.734124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec32ed-f3e5-409d-83bf-1bc973fab328 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:42.735505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 655/1329] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:42.741668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086b26-0fcb-47bc-bb14-689c42e07b63 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:42.742118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086b26-0fcb-47bc-bb14-689c42e07b63 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:42.742213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c086b26-0fcb-47bc-bb14-689c42e07b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:42.810015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1c086b26-0fcb-47bc-bb14-689c42e07b63 None None] MFA Rules not processed for user `82f38e27cd95460f986105f4aeedb6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:42.811270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c086b26-0fcb-47bc-bb14-689c42e07b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:42.906385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 675/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:42.947534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec32ed-f3e5-409d-83bf-1bc973fab328 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:42.950783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-80ec32ed-f3e5-409d-83bf-1bc973fab328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:42.989726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:43.098057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec32ed-f3e5-409d-83bf-1bc973fab328 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:43.146143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b4b46-3dff-4d49-8bf4-c5acd37fca88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:43.146780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b4b46-3dff-4d49-8bf4-c5acd37fca88 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:43.147058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b4b46-3dff-4d49-8bf4-c5acd37fca88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:43.147327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b4b46-3dff-4d49-8bf4-c5acd37fca88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:43.148928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-647b4b46-3dff-4d49-8bf4-c5acd37fca88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:43.225028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:43.256002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b4b46-3dff-4d49-8bf4-c5acd37fca88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:43.257427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b4b46-3dff-4d49-8bf4-c5acd37fca88 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:43.277368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 656/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:45.914067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbc7b4-d8d2-40b6-a7c8-706e4902e466 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:45.914623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbc7b4-d8d2-40b6-a7c8-706e4902e466 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:45.914623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbc7b4-d8d2-40b6-a7c8-706e4902e466 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:45.915905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 676/1332] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:45.921106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b07cc-4509-44df-95c4-d65504bf2394 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:45.921540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b07cc-4509-44df-95c4-d65504bf2394 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:45.921652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b07cc-4509-44df-95c4-d65504bf2394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:45.986394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-197b07cc-4509-44df-95c4-d65504bf2394 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:45.987801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-197b07cc-4509-44df-95c4-d65504bf2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:46.082425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 657/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:46.118597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbc7b4-d8d2-40b6-a7c8-706e4902e466 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:46.120133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fddbc7b4-d8d2-40b6-a7c8-706e4902e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:46.188371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:46.222110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbc7b4-d8d2-40b6-a7c8-706e4902e466 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:46.294995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b94a2-b71d-4f81-b1d9-785e7d9572bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:46.295592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b94a2-b71d-4f81-b1d9-785e7d9572bf service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:46.295869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b94a2-b71d-4f81-b1d9-785e7d9572bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:46.296133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b94a2-b71d-4f81-b1d9-785e7d9572bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:46.297755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-293b94a2-b71d-4f81-b1d9-785e7d9572bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:46.405889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b94a2-b71d-4f81-b1d9-785e7d9572bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:46.407350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b94a2-b71d-4f81-b1d9-785e7d9572bf service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:46.417178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:46.425781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 677/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:48.441006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbace15b-9e26-4def-9dea-be1a274211cb None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:48.441565 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbace15b-9e26-4def-9dea-be1a274211cb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:48.441565 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbace15b-9e26-4def-9dea-be1a274211cb None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:48.442964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 658/1335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:48.447929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627d5e0-94d8-435b-98a6-46df761e4c05 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:48.448315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627d5e0-94d8-435b-98a6-46df761e4c05 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:48.448470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627d5e0-94d8-435b-98a6-46df761e4c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:48.513229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b627d5e0-94d8-435b-98a6-46df761e4c05 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:48.514442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b627d5e0-94d8-435b-98a6-46df761e4c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:48.610537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 678/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:48.649782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbace15b-9e26-4def-9dea-be1a274211cb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:48.651359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cbace15b-9e26-4def-9dea-be1a274211cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:48.691179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:48.753194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbace15b-9e26-4def-9dea-be1a274211cb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:48.823565 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4b1c8-ba0a-4fc3-ac65-71ec0ac6b1d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:48.824151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4b1c8-ba0a-4fc3-ac65-71ec0ac6b1d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:48.824423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4b1c8-ba0a-4fc3-ac65-71ec0ac6b1d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:48.824685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4b1c8-ba0a-4fc3-ac65-71ec0ac6b1d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:48.826234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a9d4b1c8-ba0a-4fc3-ac65-71ec0ac6b1d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:48.906658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:48.932091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4b1c8-ba0a-4fc3-ac65-71ec0ac6b1d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:48.933519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4b1c8-ba0a-4fc3-ac65-71ec0ac6b1d8 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:48.952547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 659/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:51.552980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ec40c-bb7f-44b2-a08d-28522f3cb7e2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:51.553545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ec40c-bb7f-44b2-a08d-28522f3cb7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:51.553545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ec40c-bb7f-44b2-a08d-28522f3cb7e2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:51.554932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 679/1338] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:51.560099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db841b13-38c1-449e-873b-9b6ed1a78308 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:51.560536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db841b13-38c1-449e-873b-9b6ed1a78308 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:51.560643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db841b13-38c1-449e-873b-9b6ed1a78308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:51.626320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-db841b13-38c1-449e-873b-9b6ed1a78308 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:51.627741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-db841b13-38c1-449e-873b-9b6ed1a78308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:51.722856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 660/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:51.758653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ec40c-bb7f-44b2-a08d-28522f3cb7e2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:51.760205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-820ec40c-bb7f-44b2-a08d-28522f3cb7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:51.835619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:51.863870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ec40c-bb7f-44b2-a08d-28522f3cb7e2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:51.937479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5a5f5-838f-4ae5-ac2d-be1fe6912e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:51.938081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5a5f5-838f-4ae5-ac2d-be1fe6912e04 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:51.938371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5a5f5-838f-4ae5-ac2d-be1fe6912e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:51.938666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5a5f5-838f-4ae5-ac2d-be1fe6912e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:51.940315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-44c5a5f5-838f-4ae5-ac2d-be1fe6912e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:52.053349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5a5f5-838f-4ae5-ac2d-be1fe6912e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:52.054827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5a5f5-838f-4ae5-ac2d-be1fe6912e04 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:52.072615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 680/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:56.246592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57337-a6be-456b-8318-e8c5d5d23987 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:56.247143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57337-a6be-456b-8318-e8c5d5d23987 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:56.247143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57337-a6be-456b-8318-e8c5d5d23987 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:56.248518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 661/1341] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:56.253764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4bf2-9483-463a-9486-15af46b991e2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:56.254214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4bf2-9483-463a-9486-15af46b991e2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:56.255579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4bf2-9483-463a-9486-15af46b991e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:56.260693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:56.323246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-34fe4bf2-9483-463a-9486-15af46b991e2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:56.324457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-34fe4bf2-9483-463a-9486-15af46b991e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:56.423410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 681/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:56.463641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57337-a6be-456b-8318-e8c5d5d23987 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:56.465198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4be57337-a6be-456b-8318-e8c5d5d23987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:56.514226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:56.570873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57337-a6be-456b-8318-e8c5d5d23987 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:24:56.643839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e40c8-f55f-49a9-81be-f00a77a0a2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:24:56.644361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e40c8-f55f-49a9-81be-f00a77a0a2aa service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:56.644635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e40c8-f55f-49a9-81be-f00a77a0a2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:56.644904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e40c8-f55f-49a9-81be-f00a77a0a2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:56.646526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eb0e40c8-f55f-49a9-81be-f00a77a0a2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:56.738982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:24:56.755654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e40c8-f55f-49a9-81be-f00a77a0a2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:24:56.757131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e40c8-f55f-49a9-81be-f00a77a0a2aa service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:24:56.775762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 662/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:24:59.770792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d7a56-4e98-465c-aa40-3b0d8420b1ec None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:59.771338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d7a56-4e98-465c-aa40-3b0d8420b1ec None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:59.771338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d7a56-4e98-465c-aa40-3b0d8420b1ec None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:59.772853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 682/1344] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:24:59.777751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e89e0f-5669-4667-9beb-5bd41a1c744b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:24:59.778151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e89e0f-5669-4667-9beb-5bd41a1c744b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:24:59.778386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e89e0f-5669-4667-9beb-5bd41a1c744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:24:59.843219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-39e89e0f-5669-4667-9beb-5bd41a1c744b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:24:59.844413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-39e89e0f-5669-4667-9beb-5bd41a1c744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:24:59.945817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 663/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:24:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:24:59.983530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d7a56-4e98-465c-aa40-3b0d8420b1ec None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:24:59.985052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2a1d7a56-4e98-465c-aa40-3b0d8420b1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:00.073373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:00.090503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d7a56-4e98-465c-aa40-3b0d8420b1ec None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:00.164969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4413e-7326-4c5a-a0c9-33ec43def2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:00.165569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4413e-7326-4c5a-a0c9-33ec43def2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:00.165852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4413e-7326-4c5a-a0c9-33ec43def2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:00.166124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4413e-7326-4c5a-a0c9-33ec43def2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:00.167764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dcb4413e-7326-4c5a-a0c9-33ec43def2c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:00.277930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4413e-7326-4c5a-a0c9-33ec43def2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:00.279323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4413e-7326-4c5a-a0c9-33ec43def2c1 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:00.298063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 683/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:24:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:04.602184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba199f98-a7b3-417b-990c-524555345d94 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:04.602785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba199f98-a7b3-417b-990c-524555345d94 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:04.602785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba199f98-a7b3-417b-990c-524555345d94 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:04.604135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 664/1347] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:04.609449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28965-1cdc-45e5-8305-3c8f6acab6d7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:04.609912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28965-1cdc-45e5-8305-3c8f6acab6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:04.609994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28965-1cdc-45e5-8305-3c8f6acab6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:04.626511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:04.681748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1de28965-1cdc-45e5-8305-3c8f6acab6d7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:04.683005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1de28965-1cdc-45e5-8305-3c8f6acab6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:04.784523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 684/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:04.823005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba199f98-a7b3-417b-990c-524555345d94 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:04.824550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ba199f98-a7b3-417b-990c-524555345d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:04.885797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:04.935829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba199f98-a7b3-417b-990c-524555345d94 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:05.014975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e253e6b-fbf2-4b56-a24e-7ceb86cf1477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:05.015605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e253e6b-fbf2-4b56-a24e-7ceb86cf1477 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:05.015884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e253e6b-fbf2-4b56-a24e-7ceb86cf1477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:05.016154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e253e6b-fbf2-4b56-a24e-7ceb86cf1477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:05.017762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8e253e6b-fbf2-4b56-a24e-7ceb86cf1477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:05.124832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:05.129487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e253e6b-fbf2-4b56-a24e-7ceb86cf1477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:05.131051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e253e6b-fbf2-4b56-a24e-7ceb86cf1477 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:05.154272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 665/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:10.063078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579a22-0e41-4895-9af6-24caa8eeaedb None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.063566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579a22-0e41-4895-9af6-24caa8eeaedb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.064180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71579a22-0e41-4895-9af6-24caa8eeaedb None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.066165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 685/1350] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:10.083133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c261-42c4-4c35-bdc6-65d8e8a2e28b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.083622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c261-42c4-4c35-bdc6-65d8e8a2e28b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.083849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c261-42c4-4c35-bdc6-65d8e8a2e28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.195254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e447c261-42c4-4c35-bdc6-65d8e8a2e28b None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:10.196361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e447c261-42c4-4c35-bdc6-65d8e8a2e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:10.318617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 666/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:14.482744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276bbf3-e5b2-4099-b2a2-3bdde4fa4c9a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:14.483327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276bbf3-e5b2-4099-b2a2-3bdde4fa4c9a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:14.483547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276bbf3-e5b2-4099-b2a2-3bdde4fa4c9a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:14.485107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 686/1352] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:14.492046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65230506-6ad4-4d28-928c-048334586f3c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:14.492501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65230506-6ad4-4d28-928c-048334586f3c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:14.492590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65230506-6ad4-4d28-928c-048334586f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:14.548246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:14.571018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-65230506-6ad4-4d28-928c-048334586f3c None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:14.573762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-65230506-6ad4-4d28-928c-048334586f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:14.689839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 667/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:14.782146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=59 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:16.010464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b027fa-94d4-49e7-9266-e2d54d270e88 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:16.011474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b027fa-94d4-49e7-9266-e2d54d270e88 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:16.011794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b027fa-94d4-49e7-9266-e2d54d270e88 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:16.013397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 687/1354] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:16.020413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f5e2c-761e-4bc3-80ce-a8a2a774a24f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:16.020912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f5e2c-761e-4bc3-80ce-a8a2a774a24f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:16.021621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f5e2c-761e-4bc3-80ce-a8a2a774a24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:16.148769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5b0f5e2c-761e-4bc3-80ce-a8a2a774a24f None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:16.150443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5b0f5e2c-761e-4bc3-80ce-a8a2a774a24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:16.194203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:16.259235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 668/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:19.765873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e87687-16e5-4eb1-9917-587b49e9b6a0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:19.766509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e87687-16e5-4eb1-9917-587b49e9b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:19.766509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e87687-16e5-4eb1-9917-587b49e9b6a0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:19.768494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 688/1356] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:19.773966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7709-8b87-4ce4-a8b5-4a5284b5350f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:19.773966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7709-8b87-4ce4-a8b5-4a5284b5350f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:19.774549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7709-8b87-4ce4-a8b5-4a5284b5350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:19.872998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-65eb7709-8b87-4ce4-a8b5-4a5284b5350f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:19.874129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-65eb7709-8b87-4ce4-a8b5-4a5284b5350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:19.985089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 669/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:20.038985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98652-2220-4d17-99c3-9e4c5bc39648 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:20.039586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98652-2220-4d17-99c3-9e4c5bc39648 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:20.040011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98652-2220-4d17-99c3-9e4c5bc39648 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:20.044052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 689/1358] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 02:25:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:20.052397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557be2-8866-4221-af59-b716912418d1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:20.052895 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557be2-8866-4221-af59-b716912418d1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:20.053481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557be2-8866-4221-af59-b716912418d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:20.072726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:20.137566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-52557be2-8866-4221-af59-b716912418d1 None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:20.138952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-52557be2-8866-4221-af59-b716912418d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:20.261156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 670/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 02:25:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:20.269617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f98652-2220-4d17-99c3-9e4c5bc39648 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:20.271276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b4f98652-2220-4d17-99c3-9e4c5bc39648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:20.433387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f98652-2220-4d17-99c3-9e4c5bc39648 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:20.516322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:20.560886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a873a7-146e-40e2-a97f-a548114425e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:20.561487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a873a7-146e-40e2-a97f-a548114425e0 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:20.561763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a873a7-146e-40e2-a97f-a548114425e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:20.562043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a873a7-146e-40e2-a97f-a548114425e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:20.563727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d8a873a7-146e-40e2-a97f-a548114425e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:20.664195 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a873a7-146e-40e2-a97f-a548114425e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:20.665783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a873a7-146e-40e2-a97f-a548114425e0 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:20.684849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 690/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:22.935589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b7d2d-961a-4e28-b584-7a0498a09587 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.936041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b7d2d-961a-4e28-b584-7a0498a09587 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.936123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b7d2d-961a-4e28-b584-7a0498a09587 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.937623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 671/1361] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:22.942693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136771f0-b6c1-45f9-9e11-5b22c4b0f0fb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.943096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136771f0-b6c1-45f9-9e11-5b22c4b0f0fb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.943280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136771f0-b6c1-45f9-9e11-5b22c4b0f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.016840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-136771f0-b6c1-45f9-9e11-5b22c4b0f0fb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.017974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-136771f0-b6c1-45f9-9e11-5b22c4b0f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.058382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.121463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 691/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.166512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b7d2d-961a-4e28-b584-7a0498a09587 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.168274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b73b7d2d-961a-4e28-b584-7a0498a09587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.245136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.343975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b7d2d-961a-4e28-b584-7a0498a09587 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.470704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7dc56-b99f-44cc-9e6e-bd9d8ebcd34f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.471453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dc56-b99f-44cc-9e6e-bd9d8ebcd34f service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.471960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dc56-b99f-44cc-9e6e-bd9d8ebcd34f service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.472473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dc56-b99f-44cc-9e6e-bd9d8ebcd34f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.475898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c3e7dc56-b99f-44cc-9e6e-bd9d8ebcd34f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.523928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.596883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7dc56-b99f-44cc-9e6e-bd9d8ebcd34f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.598525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7dc56-b99f-44cc-9e6e-bd9d8ebcd34f service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.617436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 672/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:26.105056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167115f-ac5f-41d5-a812-2daa526104f6 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.105575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167115f-ac5f-41d5-a812-2daa526104f6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.105575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167115f-ac5f-41d5-a812-2daa526104f6 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.107015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 692/1364] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:26.113093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a5de-e7e0-4f2d-acb6-842c96b8cdf1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.113588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a5de-e7e0-4f2d-acb6-842c96b8cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.113670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a5de-e7e0-4f2d-acb6-842c96b8cdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.197326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-bc10a5de-e7e0-4f2d-acb6-842c96b8cdf1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:26.198560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bc10a5de-e7e0-4f2d-acb6-842c96b8cdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:26.303189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:26.310458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 673/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:26.354122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c4e9f-ae27-4d14-9439-5150aca1b4a0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.354417 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c4e9f-ae27-4d14-9439-5150aca1b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.354876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c4e9f-ae27-4d14-9439-5150aca1b4a0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.356193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 693/1366] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 02:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:26.364413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45cb61-fe58-40f5-89a7-2c731d38887a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.364817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45cb61-fe58-40f5-89a7-2c731d38887a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.365128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45cb61-fe58-40f5-89a7-2c731d38887a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.448870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7e45cb61-fe58-40f5-89a7-2c731d38887a None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:26.449993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7e45cb61-fe58-40f5-89a7-2c731d38887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:26.575428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 674/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 02:25:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:26.584211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c4e9f-ae27-4d14-9439-5150aca1b4a0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:26.585914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-734c4e9f-ae27-4d14-9439-5150aca1b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:26.745195 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c4e9f-ae27-4d14-9439-5150aca1b4a0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:26.767353 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:26.866930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af27ab5-8ae6-48ea-a728-fb964d0770d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:26.867531 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27ab5-8ae6-48ea-a728-fb964d0770d4 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.867805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27ab5-8ae6-48ea-a728-fb964d0770d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.868158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af27ab5-8ae6-48ea-a728-fb964d0770d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.869728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6af27ab5-8ae6-48ea-a728-fb964d0770d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:26.967988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af27ab5-8ae6-48ea-a728-fb964d0770d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:26.969716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af27ab5-8ae6-48ea-a728-fb964d0770d4 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:26.989198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 694/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:28.067206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa507b-1590-445b-8762-25de6b801f40 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:28.067738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa507b-1590-445b-8762-25de6b801f40 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:28.067738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa507b-1590-445b-8762-25de6b801f40 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:28.069928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 675/1369] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:28.074680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260164-7a05-4e73-abcd-012f9f34ea75 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:28.075082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260164-7a05-4e73-abcd-012f9f34ea75 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:28.075269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260164-7a05-4e73-abcd-012f9f34ea75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:28.109369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:28.146801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f1260164-7a05-4e73-abcd-012f9f34ea75 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:28.148178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f1260164-7a05-4e73-abcd-012f9f34ea75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:28.251731 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 695/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:28.598140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3db8-118c-490f-907a-36c95a83cc79 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:28.598140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3db8-118c-490f-907a-36c95a83cc79 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:28.598140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3db8-118c-490f-907a-36c95a83cc79 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:28.600910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 676/1371] 10.222.0.15 () {54 vars in 964 bytes} [Mon Aug 8 02:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:28.604691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972facbc-87f3-4e15-9a60-3281cd78ca47 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:28.604975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972facbc-87f3-4e15-9a60-3281cd78ca47 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:28.605267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972facbc-87f3-4e15-9a60-3281cd78ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:28.684940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-972facbc-87f3-4e15-9a60-3281cd78ca47 None None] MFA Rules not processed for user `1149adba4d37480b8d4fee34c77d340d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:28.686150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-972facbc-87f3-4e15-9a60-3281cd78ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:28.734432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:28.766525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 696/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Aug 8 02:25:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:28.774073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3db8-118c-490f-907a-36c95a83cc79 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:28.775695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ed3f3db8-118c-490f-907a-36c95a83cc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:28.832560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:28.887868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3db8-118c-490f-907a-36c95a83cc79 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:28.968962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49a613-221f-49ce-9418-9f8e6a055019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:28.970186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a613-221f-49ce-9418-9f8e6a055019 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:28.970504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a613-221f-49ce-9418-9f8e6a055019 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:28.970775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49a613-221f-49ce-9418-9f8e6a055019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:28.972577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f49a613-221f-49ce-9418-9f8e6a055019 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.083813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:29.098905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49a613-221f-49ce-9418-9f8e6a055019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:29.103107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49a613-221f-49ce-9418-9f8e6a055019 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:29.118543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 677/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:29.364108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505a9b7-41a1-4d47-8ad8-47d88a8663ac None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.364798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505a9b7-41a1-4d47-8ad8-47d88a8663ac None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.364798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505a9b7-41a1-4d47-8ad8-47d88a8663ac None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.367071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 697/1374] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:25:29.371645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d23dba-2793-4cbd-a3e8-b4669964e0ba None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.371882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d23dba-2793-4cbd-a3e8-b4669964e0ba None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.372161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d23dba-2793-4cbd-a3e8-b4669964e0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.441168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-20d23dba-2793-4cbd-a3e8-b4669964e0ba None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:29.442251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-20d23dba-2793-4cbd-a3e8-b4669964e0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.546783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 678/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:29.591393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505a9b7-41a1-4d47-8ad8-47d88a8663ac None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:29.592917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c505a9b7-41a1-4d47-8ad8-47d88a8663ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.742506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505a9b7-41a1-4d47-8ad8-47d88a8663ac None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:29.794495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:29.858832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a92187-ab2d-4086-ac28-4cc3f350e020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:29.859346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a92187-ab2d-4086-ac28-4cc3f350e020 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.859643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a92187-ab2d-4086-ac28-4cc3f350e020 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.859921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a92187-ab2d-4086-ac28-4cc3f350e020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.861512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-43a92187-ab2d-4086-ac28-4cc3f350e020 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.952964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a92187-ab2d-4086-ac28-4cc3f350e020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:29.955210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a92187-ab2d-4086-ac28-4cc3f350e020 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:29.973961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 698/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:32.161047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b3ea2-f11c-405e-95ab-a12a7eb31132 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:32.161547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b3ea2-f11c-405e-95ab-a12a7eb31132 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:32.161670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b3ea2-f11c-405e-95ab-a12a7eb31132 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:32.163037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 679/1377] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:32.168428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52b8b-7902-4fe5-9ee6-baaf9360e3c0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:32.168858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52b8b-7902-4fe5-9ee6-baaf9360e3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:32.169105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52b8b-7902-4fe5-9ee6-baaf9360e3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:32.248666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-59a52b8b-7902-4fe5-9ee6-baaf9360e3c0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:32.249854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-59a52b8b-7902-4fe5-9ee6-baaf9360e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:32.256910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:32.367059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 699/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:32.411300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b3ea2-f11c-405e-95ab-a12a7eb31132 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:32.412868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9b6b3ea2-f11c-405e-95ab-a12a7eb31132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:32.475986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:32.570150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b3ea2-f11c-405e-95ab-a12a7eb31132 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:32.704752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937f71c-9f21-4fae-ab2f-71bc414be22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:32.705354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937f71c-9f21-4fae-ab2f-71bc414be22e service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:32.705631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937f71c-9f21-4fae-ab2f-71bc414be22e service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:32.705903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937f71c-9f21-4fae-ab2f-71bc414be22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:32.711762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6937f71c-9f21-4fae-ab2f-71bc414be22e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:32.758034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:32.835499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937f71c-9f21-4fae-ab2f-71bc414be22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:32.838891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937f71c-9f21-4fae-ab2f-71bc414be22e service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:32.865273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 680/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:35.061394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c365-f7d8-48b1-834b-bdf11c64a2fb None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.061880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c365-f7d8-48b1-834b-bdf11c64a2fb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.061960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c365-f7d8-48b1-834b-bdf11c64a2fb None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.064336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 700/1380] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:35.069509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36a0d2-31b9-4381-b4b1-232bd55a5499 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.069934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36a0d2-31b9-4381-b4b1-232bd55a5499 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.070123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36a0d2-31b9-4381-b4b1-232bd55a5499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.142886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7d36a0d2-31b9-4381-b4b1-232bd55a5499 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:35.144093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7d36a0d2-31b9-4381-b4b1-232bd55a5499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.242212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:35.248207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 681/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:35.293418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137c365-f7d8-48b1-834b-bdf11c64a2fb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.295060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c137c365-f7d8-48b1-834b-bdf11c64a2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.440587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:35.451362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137c365-f7d8-48b1-834b-bdf11c64a2fb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.573820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf2970-84e5-4776-a656-e395e73e86c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.574484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf2970-84e5-4776-a656-e395e73e86c5 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.574766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf2970-84e5-4776-a656-e395e73e86c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.575033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf2970-84e5-4776-a656-e395e73e86c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.576840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-45cf2970-84e5-4776-a656-e395e73e86c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.686200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf2970-84e5-4776-a656-e395e73e86c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.687879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf2970-84e5-4776-a656-e395e73e86c5 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.698751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:35.707754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 701/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:38.199069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93580-ed23-4419-a2e0-b3e8a25daf5b None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.199593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93580-ed23-4419-a2e0-b3e8a25daf5b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.199691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93580-ed23-4419-a2e0-b3e8a25daf5b None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.201570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 682/1383] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:38.206766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69144a6c-98d2-4163-a857-d16da1e47a63 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.207338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69144a6c-98d2-4163-a857-d16da1e47a63 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.207338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69144a6c-98d2-4163-a857-d16da1e47a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.274892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-69144a6c-98d2-4163-a857-d16da1e47a63 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:38.276122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-69144a6c-98d2-4163-a857-d16da1e47a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.377922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 702/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:38.421965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e93580-ed23-4419-a2e0-b3e8a25daf5b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.423611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f8e93580-ed23-4419-a2e0-b3e8a25daf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.577295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e93580-ed23-4419-a2e0-b3e8a25daf5b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.645984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:38.702841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8481c3-2caa-4d0b-98dc-13b6d05cd894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.704613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8481c3-2caa-4d0b-98dc-13b6d05cd894 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.704980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8481c3-2caa-4d0b-98dc-13b6d05cd894 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.705397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8481c3-2caa-4d0b-98dc-13b6d05cd894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.707530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6a8481c3-2caa-4d0b-98dc-13b6d05cd894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.805649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8481c3-2caa-4d0b-98dc-13b6d05cd894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.807311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8481c3-2caa-4d0b-98dc-13b6d05cd894 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.825421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 683/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:41.461712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bba9b8-cf82-4028-b4c9-78c557e245e5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.462253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bba9b8-cf82-4028-b4c9-78c557e245e5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.462464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bba9b8-cf82-4028-b4c9-78c557e245e5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.464439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 703/1386] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:41.471560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8fdb9-b329-47e2-b260-515c616d59bb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.472071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8fdb9-b329-47e2-b260-515c616d59bb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.472189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8fdb9-b329-47e2-b260-515c616d59bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.539688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-71d8fdb9-b329-47e2-b260-515c616d59bb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:41.540893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-71d8fdb9-b329-47e2-b260-515c616d59bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:41.574078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:41.646662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 684/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:41.695189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bba9b8-cf82-4028-b4c9-78c557e245e5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.697046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-41bba9b8-cf82-4028-b4c9-78c557e245e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:41.788624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:41.860563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bba9b8-cf82-4028-b4c9-78c557e245e5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.989476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f1217-1962-4201-ab5e-6d3fc40e8f9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.990199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f1217-1962-4201-ab5e-6d3fc40e8f9a service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.990567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f1217-1962-4201-ab5e-6d3fc40e8f9a service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.990891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f1217-1962-4201-ab5e-6d3fc40e8f9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.992849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a18f1217-1962-4201-ab5e-6d3fc40e8f9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.037958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:42.095408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f1217-1962-4201-ab5e-6d3fc40e8f9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:42.097288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f1217-1962-4201-ab5e-6d3fc40e8f9a service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:42.118225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 704/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:44.255343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04826f-5858-4489-baaa-310ec6bf78fd None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.255867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04826f-5858-4489-baaa-310ec6bf78fd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.255867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04826f-5858-4489-baaa-310ec6bf78fd None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.257295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 685/1389] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:44.262232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8aebc-5fcc-4fd2-8d69-911ab5462934 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.262691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8aebc-5fcc-4fd2-8d69-911ab5462934 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.262886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8aebc-5fcc-4fd2-8d69-911ab5462934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.330549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a3f8aebc-5fcc-4fd2-8d69-911ab5462934 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:44.331770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a3f8aebc-5fcc-4fd2-8d69-911ab5462934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.429798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 705/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:44.436634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3bca2-2c19-4cc3-9418-a324fc40afd0 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.436919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3bca2-2c19-4cc3-9418-a324fc40afd0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.437803 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3bca2-2c19-4cc3-9418-a324fc40afd0 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.440375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 686/1391] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:44.445763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7a7c-3828-43b5-a320-281b456a6a93 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.446050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7a7c-3828-43b5-a320-281b456a6a93 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.446332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7a7c-3828-43b5-a320-281b456a6a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.463060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:44.475273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3bca2-2c19-4cc3-9418-a324fc40afd0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.479415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7bc3bca2-2c19-4cc3-9418-a324fc40afd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.516649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-814e7a7c-3828-43b5-a320-281b456a6a93 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:44.517780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-814e7a7c-3828-43b5-a320-281b456a6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.615457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 706/1392] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:44.624011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3bca2-2c19-4cc3-9418-a324fc40afd0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.640966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:44.673664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a988-99ba-4f2e-950d-443c3abf4d1d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.673948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a988-99ba-4f2e-950d-443c3abf4d1d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.674217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02a988-99ba-4f2e-950d-443c3abf4d1d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.676154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 707/1393] 10.222.0.15 () {54 vars in 966 bytes} [Mon Aug 8 02:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:25:44.680954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da16941-847c-4b2f-bb6d-f002649f8a20 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.681236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da16941-847c-4b2f-bb6d-f002649f8a20 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.681507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da16941-847c-4b2f-bb6d-f002649f8a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.740715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432db334-8493-4934-a2bd-257d480d5f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.741312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db334-8493-4934-a2bd-257d480d5f48 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.741585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db334-8493-4934-a2bd-257d480d5f48 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.741854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db334-8493-4934-a2bd-257d480d5f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.743490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-432db334-8493-4934-a2bd-257d480d5f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.760581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1da16941-847c-4b2f-bb6d-f002649f8a20 None None] MFA Rules not processed for user `38206a70a8b24cffabac622175335154`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:44.761835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1da16941-847c-4b2f-bb6d-f002649f8a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.797646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:44.851194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432db334-8493-4934-a2bd-257d480d5f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.853016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432db334-8493-4934-a2bd-257d480d5f48 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.859841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 708/1394] 10.222.0.15 () {58 vars in 1063 bytes} [Mon Aug 8 02:25:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:44.868204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da16941-847c-4b2f-bb6d-f002649f8a20 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.871744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 687/1395] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:44.876399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da16941-847c-4b2f-bb6d-f002649f8a20 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.961970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c1e23-d229-40f3-8db5-b92f5bd30aa7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.988874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c1e23-d229-40f3-8db5-b92f5bd30aa7 service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.988874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c1e23-d229-40f3-8db5-b92f5bd30aa7 service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.988874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c1e23-d229-40f3-8db5-b92f5bd30aa7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.988874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ff4c1e23-d229-40f3-8db5-b92f5bd30aa7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:45.076495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:45.080106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c1e23-d229-40f3-8db5-b92f5bd30aa7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:45.081563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c1e23-d229-40f3-8db5-b92f5bd30aa7 service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:45.100545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 709/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:46.022493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40011-e0e8-492c-b480-5ba49a56c197 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.022974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40011-e0e8-492c-b480-5ba49a56c197 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.023076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40011-e0e8-492c-b480-5ba49a56c197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.051323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:46.092050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1ee40011-e0e8-492c-b480-5ba49a56c197 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:46.093852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1ee40011-e0e8-492c-b480-5ba49a56c197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.181823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 688/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:46.229413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1384-2af0-488c-9db3-888b2f33982a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.229870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1384-2af0-488c-9db3-888b2f33982a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.230057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1384-2af0-488c-9db3-888b2f33982a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.231323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 710/1398] 10.222.0.15 () {54 vars in 943 bytes} [Mon Aug 8 02:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:46.237909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6dee6-053e-4dad-851a-983af333aef6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.238196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6dee6-053e-4dad-851a-983af333aef6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.238508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6dee6-053e-4dad-851a-983af333aef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.318483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4ca6dee6-053e-4dad-851a-983af333aef6 None None] MFA Rules not processed for user `481259c751234a5a936a8b0660eb8cac`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:46.319686 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4ca6dee6-053e-4dad-851a-983af333aef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.371706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:46.428308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 689/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Mon Aug 8 02:25:46 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:46.438478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af1384-2af0-488c-9db3-888b2f33982a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:46.440057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e4af1384-2af0-488c-9db3-888b2f33982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.580665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af1384-2af0-488c-9db3-888b2f33982a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:46.650527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:46.683282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a446ae-b174-4033-834b-17b93f25e3b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:46.683804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a446ae-b174-4033-834b-17b93f25e3b6 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:46.684076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a446ae-b174-4033-834b-17b93f25e3b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:46.684341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a446ae-b174-4033-834b-17b93f25e3b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:46.686720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b4a446ae-b174-4033-834b-17b93f25e3b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:46.772663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a446ae-b174-4033-834b-17b93f25e3b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:46.774259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a446ae-b174-4033-834b-17b93f25e3b6 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:46.782709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 711/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:25:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:25:47.212918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece99f1b-a0e6-48c0-84f5-e4acea124a5a None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.213515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece99f1b-a0e6-48c0-84f5-e4acea124a5a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.213515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece99f1b-a0e6-48c0-84f5-e4acea124a5a None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.214858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 690/1401] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:47.220665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4a740-8067-4f26-9853-2bfcdfe27585 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.220941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4a740-8067-4f26-9853-2bfcdfe27585 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.221217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4a740-8067-4f26-9853-2bfcdfe27585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.289773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e8d4a740-8067-4f26-9853-2bfcdfe27585 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:47.291003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8d4a740-8067-4f26-9853-2bfcdfe27585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.353017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:47.397033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 712/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:47.443761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece99f1b-a0e6-48c0-84f5-e4acea124a5a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:47.445455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ece99f1b-a0e6-48c0-84f5-e4acea124a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.665032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece99f1b-a0e6-48c0-84f5-e4acea124a5a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:47.665032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:47.753278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b285e-3871-4aaf-b2f2-1d23872a26de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:47.758387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b285e-3871-4aaf-b2f2-1d23872a26de service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.758387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b285e-3871-4aaf-b2f2-1d23872a26de service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.758387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b285e-3871-4aaf-b2f2-1d23872a26de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.758387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-681b285e-3871-4aaf-b2f2-1d23872a26de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.855007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b285e-3871-4aaf-b2f2-1d23872a26de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:47.856839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b285e-3871-4aaf-b2f2-1d23872a26de service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:47.875783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 691/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:50.061736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb5c47-ab74-440b-8fcf-fb68be33f85a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.063584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb5c47-ab74-440b-8fcf-fb68be33f85a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.064085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb5c47-ab74-440b-8fcf-fb68be33f85a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.065645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 713/1404] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:50.072009 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f20377-bdf1-4ec0-a5ca-604b3d50ffd7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.072451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f20377-bdf1-4ec0-a5ca-604b3d50ffd7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.072563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f20377-bdf1-4ec0-a5ca-604b3d50ffd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.111867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:50.146712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-63f20377-bdf1-4ec0-a5ca-604b3d50ffd7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:50.148850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-63f20377-bdf1-4ec0-a5ca-604b3d50ffd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.268584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 692/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:50.313909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb5c47-ab74-440b-8fcf-fb68be33f85a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.315498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7ecb5c47-ab74-440b-8fcf-fb68be33f85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.492477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb5c47-ab74-440b-8fcf-fb68be33f85a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.544932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:50.629277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29874eb9-74f6-4401-9267-9fb28ef4a851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.629873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29874eb9-74f6-4401-9267-9fb28ef4a851 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.630152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29874eb9-74f6-4401-9267-9fb28ef4a851 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.630460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29874eb9-74f6-4401-9267-9fb28ef4a851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.632086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-29874eb9-74f6-4401-9267-9fb28ef4a851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.735231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29874eb9-74f6-4401-9267-9fb28ef4a851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.737200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29874eb9-74f6-4401-9267-9fb28ef4a851 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.759224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 714/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:52.998424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5209aa-1ec3-4f5b-8cc6-3eceb545a063 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.998968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5209aa-1ec3-4f5b-8cc6-3eceb545a063 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.999076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5209aa-1ec3-4f5b-8cc6-3eceb545a063 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.000447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 693/1407] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:53.006106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed228d1-5a3d-4d04-96fd-71a3055acec9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:53.007507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed228d1-5a3d-4d04-96fd-71a3055acec9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:53.007939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed228d1-5a3d-4d04-96fd-71a3055acec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.084434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:53.091880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1ed228d1-5a3d-4d04-96fd-71a3055acec9 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:53.093848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1ed228d1-5a3d-4d04-96fd-71a3055acec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:53.210467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 715/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:53.252615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5209aa-1ec3-4f5b-8cc6-3eceb545a063 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:53.254143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eb5209aa-1ec3-4f5b-8cc6-3eceb545a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:53.371365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:53.416714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5209aa-1ec3-4f5b-8cc6-3eceb545a063 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:53.548101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82ea0c-265a-4340-aca4-19aceb979fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:53.548710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ea0c-265a-4340-aca4-19aceb979fb6 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:53.548989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ea0c-265a-4340-aca4-19aceb979fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:53.549260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82ea0c-265a-4340-aca4-19aceb979fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.550946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ee82ea0c-265a-4340-aca4-19aceb979fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:53.627600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:53.652285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82ea0c-265a-4340-aca4-19aceb979fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:53.653912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82ea0c-265a-4340-aca4-19aceb979fb6 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:53.676197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 694/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:03.206640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=44 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:03.673774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=14 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:10.084653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34cf36-8358-47aa-a6b1-7eb6b92195e4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.085225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34cf36-8358-47aa-a6b1-7eb6b92195e4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.085225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34cf36-8358-47aa-a6b1-7eb6b92195e4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.086562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 716/1410] 10.222.0.15 () {54 vars in 918 bytes} [Mon Aug 8 02:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:10.091590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a83fe-34b7-47d2-9ab2-5bbf3a8f9529 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.092029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a83fe-34b7-47d2-9ab2-5bbf3a8f9529 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.092857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a83fe-34b7-47d2-9ab2-5bbf3a8f9529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.168764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-977a83fe-34b7-47d2-9ab2-5bbf3a8f9529 None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:10.169971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-977a83fe-34b7-47d2-9ab2-5bbf3a8f9529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.274119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 695/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 8 02:26:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:10.289310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe713e7-83e1-44d8-af2e-f2cb0d172bee None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.289595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe713e7-83e1-44d8-af2e-f2cb0d172bee None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.289862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe713e7-83e1-44d8-af2e-f2cb0d172bee None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.291363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 717/1412] 10.222.0.15 () {54 vars in 918 bytes} [Mon Aug 8 02:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:10.302091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87753d91-ba00-4c0e-b8bf-0dc726f677ea None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.302091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87753d91-ba00-4c0e-b8bf-0dc726f677ea None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.302091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87753d91-ba00-4c0e-b8bf-0dc726f677ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.370420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:10.374057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-87753d91-ba00-4c0e-b8bf-0dc726f677ea None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:10.375391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-87753d91-ba00-4c0e-b8bf-0dc726f677ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.489559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 696/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 8 02:26:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:12.929998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a381e-fc39-4453-b179-c2d7a9027f0b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.930504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a381e-fc39-4453-b179-c2d7a9027f0b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.930612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a381e-fc39-4453-b179-c2d7a9027f0b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.931962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:12.937793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeeb74-a95e-4852-8f1c-2992079b6f40 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.938215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeeb74-a95e-4852-8f1c-2992079b6f40 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.938448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeeb74-a95e-4852-8f1c-2992079b6f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.003801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-44aeeb74-a95e-4852-8f1c-2992079b6f40 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:13.004999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-44aeeb74-a95e-4852-8f1c-2992079b6f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.058434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:13.101230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:13.145091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a381e-fc39-4453-b179-c2d7a9027f0b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.146754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-077a381e-fc39-4453-b179-c2d7a9027f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.290319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a381e-fc39-4453-b179-c2d7a9027f0b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.362788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:13.401568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac367a-fe6b-4717-896e-535c3d630385 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.402163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac367a-fe6b-4717-896e-535c3d630385 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.402470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac367a-fe6b-4717-896e-535c3d630385 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.402940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac367a-fe6b-4717-896e-535c3d630385 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.405072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b0ac367a-fe6b-4717-896e-535c3d630385 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.495992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac367a-fe6b-4717-896e-535c3d630385 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.497587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac367a-fe6b-4717-896e-535c3d630385 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.515610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 719/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:14.407363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089419-45e0-445e-a41a-943c28c5aad5 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.407926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089419-45e0-445e-a41a-943c28c5aad5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.407926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60089419-45e0-445e-a41a-943c28c5aad5 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.409330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:14.414274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d337c36-b5c7-4966-98f8-8e433075dd87 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.414717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d337c36-b5c7-4966-98f8-8e433075dd87 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.414848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d337c36-b5c7-4966-98f8-8e433075dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.489470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0d337c36-b5c7-4966-98f8-8e433075dd87 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:14.490739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0d337c36-b5c7-4966-98f8-8e433075dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.517887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:14.587879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:14.738674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60089419-45e0-445e-a41a-943c28c5aad5 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:14.740163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-60089419-45e0-445e-a41a-943c28c5aad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.884764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60089419-45e0-445e-a41a-943c28c5aad5 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.926479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:14.998815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea92e9f-4ad8-4700-a393-e42403213ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.999437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea92e9f-4ad8-4700-a393-e42403213ee3 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.999712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea92e9f-4ad8-4700-a393-e42403213ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.999978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea92e9f-4ad8-4700-a393-e42403213ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.001563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7ea92e9f-4ad8-4700-a393-e42403213ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.089254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea92e9f-4ad8-4700-a393-e42403213ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.090882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea92e9f-4ad8-4700-a393-e42403213ee3 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.109163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 699/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:15.989220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad9b0f-9094-4c6a-8bf5-7b141407a6af None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.998539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad9b0f-9094-4c6a-8bf5-7b141407a6af None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.998539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad9b0f-9094-4c6a-8bf5-7b141407a6af None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.998539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:15.998799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecdac9-df04-43af-86df-1c2ecab6c490 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.998799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecdac9-df04-43af-86df-1c2ecab6c490 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.998799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecdac9-df04-43af-86df-1c2ecab6c490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.052491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:16.073906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-43ecdac9-df04-43af-86df-1c2ecab6c490 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:16.075151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-43ecdac9-df04-43af-86df-1c2ecab6c490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.170764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:16.326441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad9b0f-9094-4c6a-8bf5-7b141407a6af None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.327968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fdad9b0f-9094-4c6a-8bf5-7b141407a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.473222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad9b0f-9094-4c6a-8bf5-7b141407a6af None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.486097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:16.587072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bca61-3ee1-4e7e-b3b2-210ffb2e92e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.587672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bca61-3ee1-4e7e-b3b2-210ffb2e92e0 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.587952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bca61-3ee1-4e7e-b3b2-210ffb2e92e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.588232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bca61-3ee1-4e7e-b3b2-210ffb2e92e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.589897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-418bca61-3ee1-4e7e-b3b2-210ffb2e92e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:16.680557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bca61-3ee1-4e7e-b3b2-210ffb2e92e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.682153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bca61-3ee1-4e7e-b3b2-210ffb2e92e0 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.700008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 722/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:17.556003 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b57e2f-0547-4989-b3f2-1d99d511d68c None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.556564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b57e2f-0547-4989-b3f2-1d99d511d68c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.556564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b57e2f-0547-4989-b3f2-1d99d511d68c None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.557886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:17.563011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48422e-32be-498c-bb05-79f6be357a22 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.563404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48422e-32be-498c-bb05-79f6be357a22 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.563551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48422e-32be-498c-bb05-79f6be357a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.583552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:17.763289 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bc48422e-32be-498c-bb05-79f6be357a22 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:17.765708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc48422e-32be-498c-bb05-79f6be357a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:17.861270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:18.012079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b57e2f-0547-4989-b3f2-1d99d511d68c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.013723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a2b57e2f-0547-4989-b3f2-1d99d511d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.121974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:18.158420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b57e2f-0547-4989-b3f2-1d99d511d68c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.288423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552fdb3-b92e-4cd4-aed6-3b09e0a35bc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.289018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fdb3-b92e-4cd4-aed6-3b09e0a35bc7 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.289296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fdb3-b92e-4cd4-aed6-3b09e0a35bc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.289559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fdb3-b92e-4cd4-aed6-3b09e0a35bc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.291196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a552fdb3-b92e-4cd4-aed6-3b09e0a35bc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.366664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:18.380660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552fdb3-b92e-4cd4-aed6-3b09e0a35bc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.382188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552fdb3-b92e-4cd4-aed6-3b09e0a35bc7 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.400356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 702/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:19.288790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de7f7-5f52-4aad-a546-db33939fdc6c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.289335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de7f7-5f52-4aad-a546-db33939fdc6c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.289335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de7f7-5f52-4aad-a546-db33939fdc6c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.290645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:19.295997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d18f2-d610-4461-948b-fad4f9ba761a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.296430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d18f2-d610-4461-948b-fad4f9ba761a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.296545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d18f2-d610-4461-948b-fad4f9ba761a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.371616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fe5d18f2-d610-4461-948b-fad4f9ba761a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:19.372813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fe5d18f2-d610-4461-948b-fad4f9ba761a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.468997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:19.619069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385de7f7-5f52-4aad-a546-db33939fdc6c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.620595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-385de7f7-5f52-4aad-a546-db33939fdc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.704610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:19.765886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385de7f7-5f52-4aad-a546-db33939fdc6c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.876390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26f096-3463-4aaa-96a5-0ac080f6b3b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.876923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26f096-3463-4aaa-96a5-0ac080f6b3b1 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.877198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26f096-3463-4aaa-96a5-0ac080f6b3b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.877467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26f096-3463-4aaa-96a5-0ac080f6b3b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.879141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ef26f096-3463-4aaa-96a5-0ac080f6b3b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.928583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:19.969272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26f096-3463-4aaa-96a5-0ac080f6b3b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.970893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26f096-3463-4aaa-96a5-0ac080f6b3b1 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.989317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 725/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:20.859698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fc230-81ec-48cb-8298-f69d94b63909 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.860260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fc230-81ec-48cb-8298-f69d94b63909 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.860260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fc230-81ec-48cb-8298-f69d94b63909 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.861600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:20.866420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2ba69-fb80-4c79-a747-b89025a07da8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.866833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2ba69-fb80-4c79-a747-b89025a07da8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.866968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2ba69-fb80-4c79-a747-b89025a07da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.945458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-95d2ba69-fb80-4c79-a747-b89025a07da8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:20.946692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-95d2ba69-fb80-4c79-a747-b89025a07da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:21.042582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:21.193751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fc230-81ec-48cb-8298-f69d94b63909 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:21.195227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2e4fc230-81ec-48cb-8298-f69d94b63909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:21.243576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:21.340437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fc230-81ec-48cb-8298-f69d94b63909 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:21.453346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09564e-1314-4a20-93eb-9ae2ec1bab68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:21.453874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09564e-1314-4a20-93eb-9ae2ec1bab68 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:21.454151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09564e-1314-4a20-93eb-9ae2ec1bab68 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:21.454452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09564e-1314-4a20-93eb-9ae2ec1bab68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:21.456039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6d09564e-1314-4a20-93eb-9ae2ec1bab68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:21.462882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:21.544066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09564e-1314-4a20-93eb-9ae2ec1bab68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:21.546238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09564e-1314-4a20-93eb-9ae2ec1bab68 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:21.564710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 705/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:22.424744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b59c2-8106-4583-a584-6b2c76f4b8ca None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:22.425323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b59c2-8106-4583-a584-6b2c76f4b8ca None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:22.425323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b59c2-8106-4583-a584-6b2c76f4b8ca None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:22.426699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:22.431589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191575ca-0f34-4422-a5a9-4f1a6f0bb81b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:22.431998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191575ca-0f34-4422-a5a9-4f1a6f0bb81b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:22.436975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191575ca-0f34-4422-a5a9-4f1a6f0bb81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:22.507543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-191575ca-0f34-4422-a5a9-4f1a6f0bb81b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:22.508741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-191575ca-0f34-4422-a5a9-4f1a6f0bb81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:22.574722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:22.603677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:22.753104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b59c2-8106-4583-a584-6b2c76f4b8ca None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:22.754670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-465b59c2-8106-4583-a584-6b2c76f4b8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:22.772174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:22.904996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b59c2-8106-4583-a584-6b2c76f4b8ca None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:22.992551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:23.018755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f921e5-8409-4c2c-b918-d4571c6dde0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.019353 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f921e5-8409-4c2c-b918-d4571c6dde0f service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.019631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f921e5-8409-4c2c-b918-d4571c6dde0f service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.019900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f921e5-8409-4c2c-b918-d4571c6dde0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.022258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62f921e5-8409-4c2c-b918-d4571c6dde0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:23.113024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f921e5-8409-4c2c-b918-d4571c6dde0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.114644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f921e5-8409-4c2c-b918-d4571c6dde0f service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.132797 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 728/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:24.012492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf39f5-24cf-4f2f-a2f1-a9568e687ea4 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:24.013018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf39f5-24cf-4f2f-a2f1-a9568e687ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:24.013018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf39f5-24cf-4f2f-a2f1-a9568e687ea4 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:24.015701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:24.020722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310ce8-81eb-44a0-b58b-5f5d245749c3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:24.021167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310ce8-81eb-44a0-b58b-5f5d245749c3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:24.021275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310ce8-81eb-44a0-b58b-5f5d245749c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:24.097413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3c310ce8-81eb-44a0-b58b-5f5d245749c3 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:24.098910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3c310ce8-81eb-44a0-b58b-5f5d245749c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:24.140963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:24.195888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:24.347999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf39f5-24cf-4f2f-a2f1-a9568e687ea4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:24.349765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-47bf39f5-24cf-4f2f-a2f1-a9568e687ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:24.493316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf39f5-24cf-4f2f-a2f1-a9568e687ea4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:24.547432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:24.727631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fdd64-876d-4c5e-b0f1-f6ab8bae6453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:24.728226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdd64-876d-4c5e-b0f1-f6ab8bae6453 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:24.728505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdd64-876d-4c5e-b0f1-f6ab8bae6453 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:24.728764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fdd64-876d-4c5e-b0f1-f6ab8bae6453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:24.730356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bd8fdd64-876d-4c5e-b0f1-f6ab8bae6453 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:24.819446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fdd64-876d-4c5e-b0f1-f6ab8bae6453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:24.821030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fdd64-876d-4c5e-b0f1-f6ab8bae6453 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:24.838716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 708/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:25.679476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a97a-fc0d-49bb-962e-34002466e86d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:25.679878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a97a-fc0d-49bb-962e-34002466e86d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:25.680018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a97a-fc0d-49bb-962e-34002466e86d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:25.681361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:25.686298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aa88d-c610-47f8-ad30-178bbf437f19 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:25.686745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aa88d-c610-47f8-ad30-178bbf437f19 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:25.686851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aa88d-c610-47f8-ad30-178bbf437f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:25.759326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6a3aa88d-c610-47f8-ad30-178bbf437f19 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:25.760498 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6a3aa88d-c610-47f8-ad30-178bbf437f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:25.762506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:25.853976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:26.007205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a97a-fc0d-49bb-962e-34002466e86d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:26.008715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1a81a97a-fc0d-49bb-962e-34002466e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:26.156952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a97a-fc0d-49bb-962e-34002466e86d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:26.183428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:26.270953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c9cb5-5213-424c-a214-7a8f72250a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:26.271556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c9cb5-5213-424c-a214-7a8f72250a5c service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:26.271830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c9cb5-5213-424c-a214-7a8f72250a5c service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:26.272098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c9cb5-5213-424c-a214-7a8f72250a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:26.273697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c21c9cb5-5213-424c-a214-7a8f72250a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:26.362981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c9cb5-5213-424c-a214-7a8f72250a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:26.364555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c9cb5-5213-424c-a214-7a8f72250a5c service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:26.382857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 731/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:28.288362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb19d-d551-4fa1-b52a-88d3f88b1ccd None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:28.288934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb19d-d551-4fa1-b52a-88d3f88b1ccd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:28.288934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166eb19d-d551-4fa1-b52a-88d3f88b1ccd None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:28.290248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:28.295682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60757d4-0835-44f7-9983-45fad4491187 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:28.296114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60757d4-0835-44f7-9983-45fad4491187 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:28.296218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60757d4-0835-44f7-9983-45fad4491187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:28.338320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:28.364677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f60757d4-0835-44f7-9983-45fad4491187 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:28.366066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f60757d4-0835-44f7-9983-45fad4491187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:28.465911 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:28.505007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eb19d-d551-4fa1-b52a-88d3f88b1ccd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:28.506630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-166eb19d-d551-4fa1-b52a-88d3f88b1ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:28.608854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166eb19d-d551-4fa1-b52a-88d3f88b1ccd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:28.640892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:28.682184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae80f76-cf7a-4ad2-b4df-32c8d45b0e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:28.682798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80f76-cf7a-4ad2-b4df-32c8d45b0e90 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:28.683067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80f76-cf7a-4ad2-b4df-32c8d45b0e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:28.683329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80f76-cf7a-4ad2-b4df-32c8d45b0e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:28.684887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3ae80f76-cf7a-4ad2-b4df-32c8d45b0e90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:28.792723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae80f76-cf7a-4ad2-b4df-32c8d45b0e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:28.794154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae80f76-cf7a-4ad2-b4df-32c8d45b0e90 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:28.812696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 711/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:30.948632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ed01-1520-44e0-83dc-c5e92a67976e None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.949139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ed01-1520-44e0-83dc-c5e92a67976e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.949250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5ed01-1520-44e0-83dc-c5e92a67976e None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:30.950559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:30.956409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e920ddf-e7b2-4b6a-b1dd-1e89bc3878d0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:30.956838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e920ddf-e7b2-4b6a-b1dd-1e89bc3878d0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:30.956934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e920ddf-e7b2-4b6a-b1dd-1e89bc3878d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:31.022787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3e920ddf-e7b2-4b6a-b1dd-1e89bc3878d0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:31.023969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3e920ddf-e7b2-4b6a-b1dd-1e89bc3878d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:31.037212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:31.121489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:31.163864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ed01-1520-44e0-83dc-c5e92a67976e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:31.165398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c4a5ed01-1520-44e0-83dc-c5e92a67976e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:31.276071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5ed01-1520-44e0-83dc-c5e92a67976e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:31.289439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:31.350866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d0d1f-17ce-4533-b695-f180deb76e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:31.351465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0d1f-17ce-4533-b695-f180deb76e76 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:31.351742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0d1f-17ce-4533-b695-f180deb76e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:31.352030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0d1f-17ce-4533-b695-f180deb76e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:31.353622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-298d0d1f-17ce-4533-b695-f180deb76e76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:31.460420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d0d1f-17ce-4533-b695-f180deb76e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:31.461833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d0d1f-17ce-4533-b695-f180deb76e76 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:31.479777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 734/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:33.884470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95afb4f-14f2-4ea1-8a7e-5ce6ab95bb96 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.885031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95afb4f-14f2-4ea1-8a7e-5ce6ab95bb96 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.885031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95afb4f-14f2-4ea1-8a7e-5ce6ab95bb96 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.886368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:33.891981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66624dd2-374f-493c-b238-f3d455bcb039 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.892381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66624dd2-374f-493c-b238-f3d455bcb039 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.892538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66624dd2-374f-493c-b238-f3d455bcb039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.942624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:33.960377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-66624dd2-374f-493c-b238-f3d455bcb039 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:33.961753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-66624dd2-374f-493c-b238-f3d455bcb039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.058648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:34.099328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95afb4f-14f2-4ea1-8a7e-5ce6ab95bb96 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.100857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f95afb4f-14f2-4ea1-8a7e-5ce6ab95bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.201593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95afb4f-14f2-4ea1-8a7e-5ce6ab95bb96 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.240655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:34.274667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb995a-96b5-4eb5-842f-59b41bcd5a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.275270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb995a-96b5-4eb5-842f-59b41bcd5a7e service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.275549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb995a-96b5-4eb5-842f-59b41bcd5a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.275820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb995a-96b5-4eb5-842f-59b41bcd5a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.277408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f9eb995a-96b5-4eb5-842f-59b41bcd5a7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.385489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb995a-96b5-4eb5-842f-59b41bcd5a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.386966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb995a-96b5-4eb5-842f-59b41bcd5a7e service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.404572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 714/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:36.520678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c489c6-6763-4a4d-bc09-5127bc084337 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.521216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c489c6-6763-4a4d-bc09-5127bc084337 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.521216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c489c6-6763-4a4d-bc09-5127bc084337 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.522603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:36.527771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb67a2-e627-4484-86ec-0820f2e25501 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.528217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb67a2-e627-4484-86ec-0820f2e25501 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.529554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb67a2-e627-4484-86ec-0820f2e25501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.593952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9bcb67a2-e627-4484-86ec-0820f2e25501 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:36.595187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9bcb67a2-e627-4484-86ec-0820f2e25501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:36.615877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:36.691489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:36.730732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c489c6-6763-4a4d-bc09-5127bc084337 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:36.732820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-67c489c6-6763-4a4d-bc09-5127bc084337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:36.834154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c489c6-6763-4a4d-bc09-5127bc084337 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:36.854604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:36.907582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7b69a-ff0b-48dc-a6ba-afde92d5ca72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:36.908180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7b69a-ff0b-48dc-a6ba-afde92d5ca72 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.908460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7b69a-ff0b-48dc-a6ba-afde92d5ca72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.908732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7b69a-ff0b-48dc-a6ba-afde92d5ca72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.910312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c2e7b69a-ff0b-48dc-a6ba-afde92d5ca72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:37.020897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7b69a-ff0b-48dc-a6ba-afde92d5ca72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:37.022327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7b69a-ff0b-48dc-a6ba-afde92d5ca72 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:37.040667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 737/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:39.090336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb03cb-00b9-4a30-863f-325ba85a79a3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:39.090964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb03cb-00b9-4a30-863f-325ba85a79a3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:39.091091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb03cb-00b9-4a30-863f-325ba85a79a3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:39.092406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:39.097562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb75bd-e08a-4d02-9a6a-91e66edecd58 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:39.097950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb75bd-e08a-4d02-9a6a-91e66edecd58 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:39.098341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb75bd-e08a-4d02-9a6a-91e66edecd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:39.160781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:39.163688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-06eb75bd-e08a-4d02-9a6a-91e66edecd58 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:39.164917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-06eb75bd-e08a-4d02-9a6a-91e66edecd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:39.261935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:39.301173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb03cb-00b9-4a30-863f-325ba85a79a3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:39.302723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a5cb03cb-00b9-4a30-863f-325ba85a79a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:39.405654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb03cb-00b9-4a30-863f-325ba85a79a3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:39.453719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:39.477781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257a6c9-9bfd-4efd-a890-19d4ded98618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:39.478395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257a6c9-9bfd-4efd-a890-19d4ded98618 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:39.478678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257a6c9-9bfd-4efd-a890-19d4ded98618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:39.478945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257a6c9-9bfd-4efd-a890-19d4ded98618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:39.480515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9257a6c9-9bfd-4efd-a890-19d4ded98618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:39.587789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257a6c9-9bfd-4efd-a890-19d4ded98618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:39.589202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257a6c9-9bfd-4efd-a890-19d4ded98618 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:39.609087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 717/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:41.658123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addacbbc-a552-46bc-bcb6-9d8bef7067e1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:41.658643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addacbbc-a552-46bc-bcb6-9d8bef7067e1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:41.658826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addacbbc-a552-46bc-bcb6-9d8bef7067e1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:41.660170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:41.665592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c1e3f-6a26-43e1-8f39-ad80c5847e84 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:41.666005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c1e3f-6a26-43e1-8f39-ad80c5847e84 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:41.667865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c1e3f-6a26-43e1-8f39-ad80c5847e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:41.734429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-032c1e3f-6a26-43e1-8f39-ad80c5847e84 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:41.735668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-032c1e3f-6a26-43e1-8f39-ad80c5847e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:41.765137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:41.832542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:41.871058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addacbbc-a552-46bc-bcb6-9d8bef7067e1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:41.873580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-addacbbc-a552-46bc-bcb6-9d8bef7067e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:41.977551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addacbbc-a552-46bc-bcb6-9d8bef7067e1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:42.005901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:42.052384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13c59d-6c50-4c7f-b084-6e68bde379c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:42.052979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13c59d-6c50-4c7f-b084-6e68bde379c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:42.053266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13c59d-6c50-4c7f-b084-6e68bde379c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:42.053535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13c59d-6c50-4c7f-b084-6e68bde379c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:42.055171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fc13c59d-6c50-4c7f-b084-6e68bde379c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:42.161421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13c59d-6c50-4c7f-b084-6e68bde379c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:42.162886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13c59d-6c50-4c7f-b084-6e68bde379c4 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:42.181543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 740/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:44.210110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0a7b-a794-4eab-8419-b82aa90108f4 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:44.210671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0a7b-a794-4eab-8419-b82aa90108f4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:44.210671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d0a7b-a794-4eab-8419-b82aa90108f4 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:44.212039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:44.217418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439db63-e09f-45c8-af49-9161a3858a97 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:44.217835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439db63-e09f-45c8-af49-9161a3858a97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:44.217977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439db63-e09f-45c8-af49-9161a3858a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:44.283162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0439db63-e09f-45c8-af49-9161a3858a97 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:44.284287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0439db63-e09f-45c8-af49-9161a3858a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:44.292478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:44.381391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:44.419351 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d0a7b-a794-4eab-8419-b82aa90108f4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:44.420930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ec3d0a7b-a794-4eab-8419-b82aa90108f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:44.525183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d0a7b-a794-4eab-8419-b82aa90108f4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:44.579174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:44.598238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e1c1d-099d-4545-89f1-6053f35ff400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:44.598857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e1c1d-099d-4545-89f1-6053f35ff400 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:44.599129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e1c1d-099d-4545-89f1-6053f35ff400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:44.599394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e1c1d-099d-4545-89f1-6053f35ff400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:44.600959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-113e1c1d-099d-4545-89f1-6053f35ff400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:44.706280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e1c1d-099d-4545-89f1-6053f35ff400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:44.707741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e1c1d-099d-4545-89f1-6053f35ff400 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:44.725790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 720/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:46.859170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c7d14-8912-4ac8-b104-63cabdcd4d9c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:46.859737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c7d14-8912-4ac8-b104-63cabdcd4d9c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:46.859737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c7d14-8912-4ac8-b104-63cabdcd4d9c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:46.861026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:46.866412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc675-3861-41de-bdda-cb55e8621c1b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:46.866913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc675-3861-41de-bdda-cb55e8621c1b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:46.867020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc675-3861-41de-bdda-cb55e8621c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:46.933164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6affc675-3861-41de-bdda-cb55e8621c1b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:46.935053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6affc675-3861-41de-bdda-cb55e8621c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:46.971297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:47.028591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:47.066853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c7d14-8912-4ac8-b104-63cabdcd4d9c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:47.068379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0d1c7d14-8912-4ac8-b104-63cabdcd4d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:47.172154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c7d14-8912-4ac8-b104-63cabdcd4d9c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:47.209334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:47.245757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed1dbd-548c-42fd-a271-4f303776ef27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:47.246343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed1dbd-548c-42fd-a271-4f303776ef27 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:47.247190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed1dbd-548c-42fd-a271-4f303776ef27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:47.247465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed1dbd-548c-42fd-a271-4f303776ef27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:47.249045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a0ed1dbd-548c-42fd-a271-4f303776ef27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:47.358071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed1dbd-548c-42fd-a271-4f303776ef27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:47.359520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed1dbd-548c-42fd-a271-4f303776ef27 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:47.377950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 743/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:49.378913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b03d6a-a1c6-4f67-8c1c-c768925a7010 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.379526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b03d6a-a1c6-4f67-8c1c-c768925a7010 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.379526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b03d6a-a1c6-4f67-8c1c-c768925a7010 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.380817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:49.386084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1043-30f9-4e4c-88bb-f1b65cfa1578 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.386600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1043-30f9-4e4c-88bb-f1b65cfa1578 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.386709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1043-30f9-4e4c-88bb-f1b65cfa1578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.452447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cb7c1043-30f9-4e4c-88bb-f1b65cfa1578 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:49.453664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cb7c1043-30f9-4e4c-88bb-f1b65cfa1578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.471368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:49.550362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:49.587529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b03d6a-a1c6-4f67-8c1c-c768925a7010 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.589077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-31b03d6a-a1c6-4f67-8c1c-c768925a7010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.694600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b03d6a-a1c6-4f67-8c1c-c768925a7010 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.759711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:49.766793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30584e35-12e2-4ba2-97b2-3baf37a9a31f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.770172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30584e35-12e2-4ba2-97b2-3baf37a9a31f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.770172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30584e35-12e2-4ba2-97b2-3baf37a9a31f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.770172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30584e35-12e2-4ba2-97b2-3baf37a9a31f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.770172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-30584e35-12e2-4ba2-97b2-3baf37a9a31f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.877023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30584e35-12e2-4ba2-97b2-3baf37a9a31f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.878489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30584e35-12e2-4ba2-97b2-3baf37a9a31f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.896812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 723/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:51.987776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e7127-45d4-4e49-b1c8-18e01151f1b2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.988309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e7127-45d4-4e49-b1c8-18e01151f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.988309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e7127-45d4-4e49-b1c8-18e01151f1b2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.989630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:51.994779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fd589-594b-4a2b-ace7-43fcddacf1f2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.995210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fd589-594b-4a2b-ace7-43fcddacf1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.995320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fd589-594b-4a2b-ace7-43fcddacf1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.061894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-206fd589-594b-4a2b-ace7-43fcddacf1f2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:52.063115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-206fd589-594b-4a2b-ace7-43fcddacf1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.112039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:52.160231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:52.197085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e7127-45d4-4e49-b1c8-18e01151f1b2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:52.198619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b55e7127-45d4-4e49-b1c8-18e01151f1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.304355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e7127-45d4-4e49-b1c8-18e01151f1b2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:52.351476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:52.377808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21831f1b-b985-480e-b4ff-fe71a1ddd6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:52.383413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21831f1b-b985-480e-b4ff-fe71a1ddd6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:52.383413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21831f1b-b985-480e-b4ff-fe71a1ddd6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:52.383413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21831f1b-b985-480e-b4ff-fe71a1ddd6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.383413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-21831f1b-b985-480e-b4ff-fe71a1ddd6c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.488237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21831f1b-b985-480e-b4ff-fe71a1ddd6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:52.489677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21831f1b-b985-480e-b4ff-fe71a1ddd6c9 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:52.508766 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 746/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:02.158498 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=22 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:02.506544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=66 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:11.990549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a6824-6852-46b6-bfdc-0b1774fa7b52 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:11.991114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a6824-6852-46b6-bfdc-0b1774fa7b52 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:11.991114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a6824-6852-46b6-bfdc-0b1774fa7b52 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:11.992451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 725/1471] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:11.998660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcbff-f0ff-42b0-a29c-9fd9b74e6893 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:11.999061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcbff-f0ff-42b0-a29c-9fd9b74e6893 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:11.999203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbcbff-f0ff-42b0-a29c-9fd9b74e6893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:12.065564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4bcbcbff-f0ff-42b0-a29c-9fd9b74e6893 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:12.066819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4bcbcbff-f0ff-42b0-a29c-9fd9b74e6893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:12.161454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 747/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:12.244508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976209-e11d-4208-b3a2-13e05ca31bba None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:12.244788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976209-e11d-4208-b3a2-13e05ca31bba None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:12.245058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14976209-e11d-4208-b3a2-13e05ca31bba None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:12.247153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 726/1473] 10.222.0.15 () {54 vars in 966 bytes} [Mon Aug 8 02:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:12.251790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3accc45-b95f-4c2c-b3a2-b08afbe67760 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:12.252086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3accc45-b95f-4c2c-b3a2-b08afbe67760 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:12.252363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3accc45-b95f-4c2c-b3a2-b08afbe67760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:12.327871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f3accc45-b95f-4c2c-b3a2-b08afbe67760 None None] MFA Rules not processed for user `38206a70a8b24cffabac622175335154`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:12.329083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f3accc45-b95f-4c2c-b3a2-b08afbe67760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:12.349302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:12.405621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 748/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Mon Aug 8 02:27:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:12.413091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976209-e11d-4208-b3a2-13e05ca31bba None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:12.414708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-14976209-e11d-4208-b3a2-13e05ca31bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:12.515814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14976209-e11d-4208-b3a2-13e05ca31bba None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:12.586917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c742ad-4f78-49e9-96d2-8e1af49d6b05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:12.587499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c742ad-4f78-49e9-96d2-8e1af49d6b05 service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:12.587774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c742ad-4f78-49e9-96d2-8e1af49d6b05 service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:12.588041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c742ad-4f78-49e9-96d2-8e1af49d6b05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:12.590118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c6c742ad-4f78-49e9-96d2-8e1af49d6b05 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:12.642531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:12.696303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c742ad-4f78-49e9-96d2-8e1af49d6b05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:12.697738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c742ad-4f78-49e9-96d2-8e1af49d6b05 service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:12.716573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 727/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:14.793296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5f899-6326-4809-b0d1-020aec4d2556 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:14.793880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5f899-6326-4809-b0d1-020aec4d2556 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:14.793880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5f899-6326-4809-b0d1-020aec4d2556 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:14.795188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 749/1476] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:27:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:14.800388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0064b-533c-4211-ba4d-0c6f71cdcc2f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:14.800836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0064b-533c-4211-ba4d-0c6f71cdcc2f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:14.800954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0064b-533c-4211-ba4d-0c6f71cdcc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:14.865872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1fe0064b-533c-4211-ba4d-0c6f71cdcc2f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:14.867113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1fe0064b-533c-4211-ba4d-0c6f71cdcc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:14.965260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 728/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:14.989209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5f899-6326-4809-b0d1-020aec4d2556 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:14.990760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d8c5f899-6326-4809-b0d1-020aec4d2556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:15.115706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5f899-6326-4809-b0d1-020aec4d2556 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:15.164404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:15.215154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775c44b-9cd2-4245-80ad-a87df79f2611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:15.215744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775c44b-9cd2-4245-80ad-a87df79f2611 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:15.216025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775c44b-9cd2-4245-80ad-a87df79f2611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:15.216298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775c44b-9cd2-4245-80ad-a87df79f2611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:15.216932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775c44b-9cd2-4245-80ad-a87df79f2611 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:15.218667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-e775c44b-9cd2-4245-80ad-a87df79f2611 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:27:15.218667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:27:15.218667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:27:15.219808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775c44b-9cd2-4245-80ad-a87df79f2611 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:15.238555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 750/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 02:27:14 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:17.253409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def6f5-4441-4cc4-a57b-c6852ec6fedb None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.253964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def6f5-4441-4cc4-a57b-c6852ec6fedb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.253964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def6f5-4441-4cc4-a57b-c6852ec6fedb None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.255392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 729/1479] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:17.260458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6faa-505e-4555-b3d5-839f91d28456 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.261616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6faa-505e-4555-b3d5-839f91d28456 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.261616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef6faa-505e-4555-b3d5-839f91d28456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.327528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-73ef6faa-505e-4555-b3d5-839f91d28456 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:17.328753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-73ef6faa-505e-4555-b3d5-839f91d28456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:17.425422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 751/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:17.468905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68def6f5-4441-4cc4-a57b-c6852ec6fedb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:17.470491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-68def6f5-4441-4cc4-a57b-c6852ec6fedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:17.480325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:17.613933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68def6f5-4441-4cc4-a57b-c6852ec6fedb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:17.692077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:17.725549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ff71a-f76a-43ed-8299-ba88b9f261bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:17.726148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ff71a-f76a-43ed-8299-ba88b9f261bc service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.726452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ff71a-f76a-43ed-8299-ba88b9f261bc service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.726723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ff71a-f76a-43ed-8299-ba88b9f261bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.728306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b65ff71a-f76a-43ed-8299-ba88b9f261bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:17.815568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ff71a-f76a-43ed-8299-ba88b9f261bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:17.817154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ff71a-f76a-43ed-8299-ba88b9f261bc service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:17.834550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 730/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:20.009282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cc54-5fea-437a-994f-88c2ce5c8725 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:20.009840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cc54-5fea-437a-994f-88c2ce5c8725 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:20.009840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cc54-5fea-437a-994f-88c2ce5c8725 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:20.011412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 752/1482] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:20.016477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11a2f6-a0a3-45d0-a38d-1b2fb45a22af None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:20.016940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11a2f6-a0a3-45d0-a38d-1b2fb45a22af None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:20.017045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11a2f6-a0a3-45d0-a38d-1b2fb45a22af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:20.082748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3a11a2f6-a0a3-45d0-a38d-1b2fb45a22af None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:20.083948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3a11a2f6-a0a3-45d0-a38d-1b2fb45a22af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:20.130123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:20.179790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 731/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:20.215991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2cc54-5fea-437a-994f-88c2ce5c8725 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:20.217509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e5a2cc54-5fea-437a-994f-88c2ce5c8725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:20.247948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:20.331027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2cc54-5fea-437a-994f-88c2ce5c8725 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:20.421254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e23be-e2ca-4c09-91dd-75f6bdef2af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:20.421858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e23be-e2ca-4c09-91dd-75f6bdef2af6 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:20.422135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e23be-e2ca-4c09-91dd-75f6bdef2af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:20.422455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e23be-e2ca-4c09-91dd-75f6bdef2af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:20.426051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ee2e23be-e2ca-4c09-91dd-75f6bdef2af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:20.513296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:20.552977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e23be-e2ca-4c09-91dd-75f6bdef2af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:20.555739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e23be-e2ca-4c09-91dd-75f6bdef2af6 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:20.576823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 753/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:22.663339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efa15b-69ff-4b54-835f-8ebd23e6d822 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:22.663873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efa15b-69ff-4b54-835f-8ebd23e6d822 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:22.663873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efa15b-69ff-4b54-835f-8ebd23e6d822 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:22.665140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 732/1485] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:22.670861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4038b-1887-4b99-b358-2b48593618b2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:22.671310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4038b-1887-4b99-b358-2b48593618b2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:22.671422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4038b-1887-4b99-b358-2b48593618b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:22.736736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b8d4038b-1887-4b99-b358-2b48593618b2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:22.737926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b8d4038b-1887-4b99-b358-2b48593618b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:22.833963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 754/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:22.873510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efa15b-69ff-4b54-835f-8ebd23e6d822 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:22.876187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-99efa15b-69ff-4b54-835f-8ebd23e6d822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:22.980529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efa15b-69ff-4b54-835f-8ebd23e6d822 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:23.050876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:23.052656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54688d-1a75-4714-a393-46eadbe1bf4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:23.053252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54688d-1a75-4714-a393-46eadbe1bf4c service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:23.053529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54688d-1a75-4714-a393-46eadbe1bf4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:23.053796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54688d-1a75-4714-a393-46eadbe1bf4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:23.055411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2e54688d-1a75-4714-a393-46eadbe1bf4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:23.162493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54688d-1a75-4714-a393-46eadbe1bf4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:23.163923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54688d-1a75-4714-a393-46eadbe1bf4c service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:23.182025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 733/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:25.355227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ee58c-865f-4f95-a05d-2abddd83b0a8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.355775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ee58c-865f-4f95-a05d-2abddd83b0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.355884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ee58c-865f-4f95-a05d-2abddd83b0a8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.357215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 755/1488] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:25.362294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9accb9-e619-4130-ad9a-bb3a915526fb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.362713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9accb9-e619-4130-ad9a-bb3a915526fb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.362870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9accb9-e619-4130-ad9a-bb3a915526fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.426863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-be9accb9-e619-4130-ad9a-bb3a915526fb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:25.428048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-be9accb9-e619-4130-ad9a-bb3a915526fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.478752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:25.522945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 734/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:25.563852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ee58c-865f-4f95-a05d-2abddd83b0a8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:25.565380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a42ee58c-865f-4f95-a05d-2abddd83b0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.602391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:25.671778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ee58c-865f-4f95-a05d-2abddd83b0a8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:25.744337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799d475-763b-4c42-b95d-13da8b76916b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:25.745322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799d475-763b-4c42-b95d-13da8b76916b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.745600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799d475-763b-4c42-b95d-13da8b76916b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.745867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799d475-763b-4c42-b95d-13da8b76916b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.747472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6799d475-763b-4c42-b95d-13da8b76916b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.826429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:25.856547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799d475-763b-4c42-b95d-13da8b76916b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:25.857967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799d475-763b-4c42-b95d-13da8b76916b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:25.876604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 756/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:28.269903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932d5e7-e0ac-4110-a5bb-29ce979148d4 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:28.270481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932d5e7-e0ac-4110-a5bb-29ce979148d4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:28.270481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932d5e7-e0ac-4110-a5bb-29ce979148d4 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:28.271833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 735/1491] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 02:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:27:28.277274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d680d3c-c30d-4bc3-b4cc-4d6d375710a0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:28.277683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d680d3c-c30d-4bc3-b4cc-4d6d375710a0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:28.277832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d680d3c-c30d-4bc3-b4cc-4d6d375710a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:28.343094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6d680d3c-c30d-4bc3-b4cc-4d6d375710a0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:28.344303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6d680d3c-c30d-4bc3-b4cc-4d6d375710a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:28.441352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 757/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 02:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:28.482632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932d5e7-e0ac-4110-a5bb-29ce979148d4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:28.484401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5932d5e7-e0ac-4110-a5bb-29ce979148d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:28.590363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932d5e7-e0ac-4110-a5bb-29ce979148d4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:28.667267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd04f6-f5aa-41b7-82cf-2e6ab108ae0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:28.667944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd04f6-f5aa-41b7-82cf-2e6ab108ae0f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:28.668221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd04f6-f5aa-41b7-82cf-2e6ab108ae0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:28.668489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd04f6-f5aa-41b7-82cf-2e6ab108ae0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:28.670424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-70bd04f6-f5aa-41b7-82cf-2e6ab108ae0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:28.679646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:28.779031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd04f6-f5aa-41b7-82cf-2e6ab108ae0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:28.780468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd04f6-f5aa-41b7-82cf-2e6ab108ae0f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:28.798746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 736/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:38.439609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=82 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:38.796423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=50 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:46.821443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:46.822068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:46.822068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:46.896683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:46.897950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:46.997533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 758/1494] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:47.005906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce54ff2-6b14-4277-a587-d9451b6a7cf0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:47.006431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce54ff2-6b14-4277-a587-d9451b6a7cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:47.006540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce54ff2-6b14-4277-a587-d9451b6a7cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:47.082814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4ce54ff2-6b14-4277-a587-d9451b6a7cf0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:47.083938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4ce54ff2-6b14-4277-a587-d9451b6a7cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:47.178594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 737/1495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:47.183575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:47.184805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:47.196349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb91f64-fa93-494d-afe5-69aab6bdddd2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:47.256629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:47.290913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:47.291506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:47.291782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:47.292050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:47.292684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:47.292980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:47.293976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:47.293976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:28:47.293976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:28:47.294882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:47.305828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 759/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:47.314064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce54ff2-6b14-4277-a587-d9451b6a7cf0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:47.323799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce54ff2-6b14-4277-a587-d9451b6a7cf0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:47.383994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:47.417698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:47.418293 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:47.418597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:47.418865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:47.419472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:47.421196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:47.421196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Aug 08 02:28:47.421196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: Aug 08 02:28:47.422396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:47.539373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 738/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:47 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:47.543706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:47.545212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:47.579073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba136-91f4-453b-b229-5bf5d6cebd51 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:47.672133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:47.672639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:47.672915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:47.673186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:47.673810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:47.674839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:47.674839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:28:47.674839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:28:47.676084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:47.771836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 760/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:47 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:28:47.776064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:47.777567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:47.838400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21377d00-3dd2-4df1-9bcf-964cd837abc5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:47.919138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:47.929747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:47.930375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:47.930651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:47.930911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:47.931539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:47.932545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:47.932545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Aug 08 02:28:47.932545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: Aug 08 02:28:47.933418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:47.946428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 739/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:47.950499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:47.952007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:47.963600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548cf08-f722-41db-9e51-f657e9a2bda9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:47.988726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:48.058801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a891a8-32e1-4271-8e33-70506048868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:48.059408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a891a8-32e1-4271-8e33-70506048868e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.059682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a891a8-32e1-4271-8e33-70506048868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.059952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a891a8-32e1-4271-8e33-70506048868e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.060578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a891a8-32e1-4271-8e33-70506048868e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:48.061573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-a9a891a8-32e1-4271-8e33-70506048868e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:48.061573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:28:48.061573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:28:48.062474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a891a8-32e1-4271-8e33-70506048868e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:48.074318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 761/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:48.078558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:48.080080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:48.113127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fed7c-39a7-425f-a827-fdf199c30530 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:48.206276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f348a6-2841-42f1-bdcb-193e9d3c9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:48.206880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f348a6-2841-42f1-bdcb-193e9d3c9ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.207161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f348a6-2841-42f1-bdcb-193e9d3c9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.207429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f348a6-2841-42f1-bdcb-193e9d3c9ea7 admin admin] PATH_INFO: `/v3/projects/6704a76dc1834f79ad128246eeb79a4a/users/d4a63be5f12a435e85b1d346ca6aa537/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.229617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f348a6-2841-42f1-bdcb-193e9d3c9ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6704a76dc1834f79ad128246eeb79a4a, user_id=d4a63be5f12a435e85b1d346ca6aa537, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:48.230678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-75f348a6-2841-42f1-bdcb-193e9d3c9ea7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:48.230678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Aug 08 02:28:48.230678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: Aug 08 02:28:48.232599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f348a6-2841-42f1-bdcb-193e9d3c9ea7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:48.324918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 740/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:48 2022] PUT /identity/v3/projects/6704a76dc1834f79ad128246eeb79a4a/users/d4a63be5f12a435e85b1d346ca6aa537/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:48.337623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee18a4-90fd-4962-9556-d5b126f3ef53 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.337913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee18a4-90fd-4962-9556-d5b126f3ef53 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.338189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee18a4-90fd-4962-9556-d5b126f3ef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.394570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7bee18a4-90fd-4962-9556-d5b126f3ef53 None None] MFA Rules not processed for user `d4a63be5f12a435e85b1d346ca6aa537`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:48.395828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7bee18a4-90fd-4962-9556-d5b126f3ef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:48.473643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 762/1502] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:28:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:48.487636 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab84a9-17b4-4ba3-807c-ba96d2aa90f4 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.487924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab84a9-17b4-4ba3-807c-ba96d2aa90f4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.488198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab84a9-17b4-4ba3-807c-ba96d2aa90f4 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.489583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 741/1503] 10.222.0.15 () {54 vars in 964 bytes} [Mon Aug 8 02:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:28:48.494597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f376-da81-4595-bfba-8afd2cd9a673 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.494882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f376-da81-4595-bfba-8afd2cd9a673 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.495153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f376-da81-4595-bfba-8afd2cd9a673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.525909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:48.568680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1c89f376-da81-4595-bfba-8afd2cd9a673 None None] MFA Rules not processed for user `1149adba4d37480b8d4fee34c77d340d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:48.569910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c89f376-da81-4595-bfba-8afd2cd9a673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:48.644183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 763/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Aug 8 02:28:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:48.652488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab84a9-17b4-4ba3-807c-ba96d2aa90f4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:48.653995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-45ab84a9-17b4-4ba3-807c-ba96d2aa90f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:48.702913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:48.755374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab84a9-17b4-4ba3-807c-ba96d2aa90f4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:48.827963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:48.828558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.828836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.829106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.830752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:48.927875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:48.929619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:48.933088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:48.947415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 742/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:48.993602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5002aa-2e95-4256-9010-9cd61b73b803 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.994023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5002aa-2e95-4256-9010-9cd61b73b803 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.994142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5002aa-2e95-4256-9010-9cd61b73b803 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.995427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 764/1506] 10.222.0.15 () {52 vars in 830 bytes} [Mon Aug 8 02:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:28:48.998945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:49.000139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:49.097962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854e674-29ae-4aa0-bf9f-7dccc7a86484 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:49.189253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:49.189873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:49.190153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:49.190457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] PATH_INFO: `/v3/users/d4a63be5f12a435e85b1d346ca6aa537` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:49.202291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a63be5f12a435e85b1d346ca6aa537)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:49.203354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:49.203354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Aug 08 02:28:49.203354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: Aug 08 02:28:49.204562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:49.214707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:49.378963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a63be5f12a435e85b1d346ca6aa537'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:49.439394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a63be5f12a435e85b1d346ca6aa537'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:49.462664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a63be5f12a435e85b1d346ca6aa537'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:49.477285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a63be5f12a435e85b1d346ca6aa537'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:49.477827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a63be5f12a435e85b1d346ca6aa537'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:49.483536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 743/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:48 2022] DELETE /identity/v3/users/d4a63be5f12a435e85b1d346ca6aa537 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:49.487927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:49.488220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:49.488495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:49.544714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:49.545923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:49.620460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:49.633058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 765/1508] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:49.643229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:49.644445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:49.744156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fdaa8-29a5-43f5-b838-b62ef04737c7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:49.798416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:49.817203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181f4d8-26c5-4f48-af30-7b6d4938e435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:49.817817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181f4d8-26c5-4f48-af30-7b6d4938e435 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:49.818096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181f4d8-26c5-4f48-af30-7b6d4938e435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:49.818380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181f4d8-26c5-4f48-af30-7b6d4938e435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:49.820481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9181f4d8-26c5-4f48-af30-7b6d4938e435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:49.928518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181f4d8-26c5-4f48-af30-7b6d4938e435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:49.930560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181f4d8-26c5-4f48-af30-7b6d4938e435 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:49.949072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 744/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:50.436107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:50.437337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:50.449751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7cac-720a-4c51-b13d-71c54de0ed89 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:50.548780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:50.549410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:50.549690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:50.549959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] PATH_INFO: `/v3/projects/6704a76dc1834f79ad128246eeb79a4a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:50.560675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6704a76dc1834f79ad128246eeb79a4a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:50.561695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:28:50.561695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:28:50.561695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:28:50.562931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:50.678014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6704a76dc1834f79ad128246eeb79a4a', 'request_id': 'req-42418295-a64e-4419-be89-39a75fa321b3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:50.747756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6704a76dc1834f79ad128246eeb79a4a', 'request_id': 'req-42418295-a64e-4419-be89-39a75fa321b3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:50.761135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6704a76dc1834f79ad128246eeb79a4a', 'request_id': 'req-42418295-a64e-4419-be89-39a75fa321b3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:50.783553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] The token cache is being invalidate because project 6704a76dc1834f79ad128246eeb79a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:50.783839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:50.784359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-42418295-a64e-4419-be89-39a75fa321b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:50.785657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 766/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:50 2022] DELETE /identity/v3/projects/6704a76dc1834f79ad128246eeb79a4a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:59.946844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=62 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:00.781664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:17.971600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422458-24fb-48a9-83ff-5c3933b956ac None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:17.972151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422458-24fb-48a9-83ff-5c3933b956ac None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:17.972151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422458-24fb-48a9-83ff-5c3933b956ac None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:17.974233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:17.979047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b828a08-bebe-4e63-a638-5d258962b970 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:17.979494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b828a08-bebe-4e63-a638-5d258962b970 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:17.979611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b828a08-bebe-4e63-a638-5d258962b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.066817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3b828a08-bebe-4e63-a638-5d258962b970 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:18.066817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3b828a08-bebe-4e63-a638-5d258962b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:18.170659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:18.326388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10422458-24fb-48a9-83ff-5c3933b956ac None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:18.332605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-10422458-24fb-48a9-83ff-5c3933b956ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:18.445299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10422458-24fb-48a9-83ff-5c3933b956ac None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:18.450694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6704a71-2867-4015-aa0f-4b7596b1ee38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:18.451284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6704a71-2867-4015-aa0f-4b7596b1ee38 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:18.451558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6704a71-2867-4015-aa0f-4b7596b1ee38 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:18.451826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6704a71-2867-4015-aa0f-4b7596b1ee38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:18.460336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6704a71-2867-4015-aa0f-4b7596b1ee38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:18.461901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6704a71-2867-4015-aa0f-4b7596b1ee38 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:18.471305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 746/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:19.300341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf51af7-3fba-4c14-871b-cee57e466aa1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.301457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf51af7-3fba-4c14-871b-cee57e466aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.304461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf51af7-3fba-4c14-871b-cee57e466aa1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.304461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:19.307808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb006-5f39-4415-a479-cd25c56fdce0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.308282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb006-5f39-4415-a479-cd25c56fdce0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.308406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb006-5f39-4415-a479-cd25c56fdce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.329699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-63edb006-5f39-4415-a479-cd25c56fdce0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:19.334033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-63edb006-5f39-4415-a479-cd25c56fdce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:19.338950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:19.495004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf51af7-3fba-4c14-871b-cee57e466aa1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:19.509425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf51af7-3fba-4c14-871b-cee57e466aa1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:19.514838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d34c5-1420-4244-8197-9e88c2ff4e10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:19.515416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d34c5-1420-4244-8197-9e88c2ff4e10 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:19.515691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d34c5-1420-4244-8197-9e88c2ff4e10 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:19.515959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d34c5-1420-4244-8197-9e88c2ff4e10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:19.524620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d34c5-1420-4244-8197-9e88c2ff4e10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:19.526172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d34c5-1420-4244-8197-9e88c2ff4e10 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:19.536441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 769/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:20.538897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e16e-21df-402b-8ac3-afc8d089902e None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:20.539367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e16e-21df-402b-8ac3-afc8d089902e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:20.539476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e16e-21df-402b-8ac3-afc8d089902e None None] PATH_INFO: `/v3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:20.540719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 748/1517] 10.222.0.48 () {54 vars in 925 bytes} [Mon Aug 8 02:30:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:30:20.546869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87893d-6eb3-4e1d-8911-d581e0753cf8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:20.547292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87893d-6eb3-4e1d-8911-d581e0753cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:20.547390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87893d-6eb3-4e1d-8911-d581e0753cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:20.609282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ec87893d-6eb3-4e1d-8911-d581e0753cf8 None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:20.610572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ec87893d-6eb3-4e1d-8911-d581e0753cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:20.629142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:20.689004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 770/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 8 02:30:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:24.778583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af4cd-791b-4fe5-97a9-f2e278d1e089 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:24.779130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af4cd-791b-4fe5-97a9-f2e278d1e089 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:24.779130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af4cd-791b-4fe5-97a9-f2e278d1e089 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:24.780371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 749/1519] 10.222.0.15 () {54 vars in 961 bytes} [Mon Aug 8 02:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:24.785729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71f405-8850-4a3f-8d58-87ad7253152a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:24.786270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71f405-8850-4a3f-8d58-87ad7253152a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:24.786403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71f405-8850-4a3f-8d58-87ad7253152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:24.805868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bb71f405-8850-4a3f-8d58-87ad7253152a None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:24.807081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bb71f405-8850-4a3f-8d58-87ad7253152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:24.815510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 771/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Aug 8 02:30:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:24.823652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3af4cd-791b-4fe5-97a9-f2e278d1e089 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:24.837005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3af4cd-791b-4fe5-97a9-f2e278d1e089 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:24.842107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70523390-c068-4518-aa29-e25b5dd0c368 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:24.842717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523390-c068-4518-aa29-e25b5dd0c368 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:24.842996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523390-c068-4518-aa29-e25b5dd0c368 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:24.843268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523390-c068-4518-aa29-e25b5dd0c368 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:24.851662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70523390-c068-4518-aa29-e25b5dd0c368 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:24.853241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70523390-c068-4518-aa29-e25b5dd0c368 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:24.862888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 750/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:30.747293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f76f6-181c-4884-92f7-17c9ebe87487 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.747850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f76f6-181c-4884-92f7-17c9ebe87487 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.747850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f76f6-181c-4884-92f7-17c9ebe87487 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.749190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:30.753965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5d33-6b6f-4215-96c3-5044bcf445d5 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.754381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5d33-6b6f-4215-96c3-5044bcf445d5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.754554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f5d33-6b6f-4215-96c3-5044bcf445d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.774032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-480f5d33-6b6f-4215-96c3-5044bcf445d5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:30.775251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-480f5d33-6b6f-4215-96c3-5044bcf445d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:30.784539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:30.939532 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f76f6-181c-4884-92f7-17c9ebe87487 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.954707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f76f6-181c-4884-92f7-17c9ebe87487 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.960194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9bcbb-83ff-47a9-8731-d4a275398fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.960781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9bcbb-83ff-47a9-8731-d4a275398fab service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.961057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9bcbb-83ff-47a9-8731-d4a275398fab service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.961325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9bcbb-83ff-47a9-8731-d4a275398fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.969622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9bcbb-83ff-47a9-8731-d4a275398fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.971281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9bcbb-83ff-47a9-8731-d4a275398fab service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.980800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 773/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:31.803254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eec77-37f8-4b82-a175-988cbee8e537 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:31.803808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eec77-37f8-4b82-a175-988cbee8e537 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:31.803808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eec77-37f8-4b82-a175-988cbee8e537 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:31.805130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:31.809788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157c87c-d9fc-45b3-9b81-2908182ecf63 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:31.810165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157c87c-d9fc-45b3-9b81-2908182ecf63 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:31.810338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157c87c-d9fc-45b3-9b81-2908182ecf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:31.829832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a157c87c-d9fc-45b3-9b81-2908182ecf63 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:31.831052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a157c87c-d9fc-45b3-9b81-2908182ecf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:31.839550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:31.992039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4eec77-37f8-4b82-a175-988cbee8e537 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:32.007431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4eec77-37f8-4b82-a175-988cbee8e537 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:32.013045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06321-c9af-4da2-8052-5be25f0befde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:32.013632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06321-c9af-4da2-8052-5be25f0befde service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:32.013910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06321-c9af-4da2-8052-5be25f0befde service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:32.014180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06321-c9af-4da2-8052-5be25f0befde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:32.022880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06321-c9af-4da2-8052-5be25f0befde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:32.024513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06321-c9af-4da2-8052-5be25f0befde service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:32.034056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 753/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:35.827135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fdde3-ddb5-4c96-854a-5eef77df5dbd None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:35.827683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fdde3-ddb5-4c96-854a-5eef77df5dbd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:35.827683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fdde3-ddb5-4c96-854a-5eef77df5dbd None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:35.829057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:35.834364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15764f-12a2-4f12-a670-f882d7f87853 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:35.834865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15764f-12a2-4f12-a670-f882d7f87853 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:35.834981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15764f-12a2-4f12-a670-f882d7f87853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:35.854903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0f15764f-12a2-4f12-a670-f882d7f87853 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:35.856088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0f15764f-12a2-4f12-a670-f882d7f87853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:35.864025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:35.940365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fdde3-ddb5-4c96-854a-5eef77df5dbd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:35.942082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b7fdde3-ddb5-4c96-854a-5eef77df5dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:36.022368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fdde3-ddb5-4c96-854a-5eef77df5dbd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:36.026108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f44d8-1245-477b-9fce-d4c70878f6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:36.026740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f44d8-1245-477b-9fce-d4c70878f6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:36.027017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f44d8-1245-477b-9fce-d4c70878f6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:36.027287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f44d8-1245-477b-9fce-d4c70878f6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:36.037117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f44d8-1245-477b-9fce-d4c70878f6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:36.039983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f44d8-1245-477b-9fce-d4c70878f6c2 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:36.048570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 776/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:38.066038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763d114-e452-4608-a590-c81b2e2cbb2c None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:38.066610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763d114-e452-4608-a590-c81b2e2cbb2c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:38.066610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763d114-e452-4608-a590-c81b2e2cbb2c None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:38.069034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:38.074568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2db4-0ac2-4704-9711-b3df2faed391 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:38.075048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2db4-0ac2-4704-9711-b3df2faed391 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:38.075149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2db4-0ac2-4704-9711-b3df2faed391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:38.095698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1d6a2db4-0ac2-4704-9711-b3df2faed391 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:38.096895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1d6a2db4-0ac2-4704-9711-b3df2faed391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:38.105030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:38.142444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763d114-e452-4608-a590-c81b2e2cbb2c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:38.144122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9763d114-e452-4608-a590-c81b2e2cbb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:38.158176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763d114-e452-4608-a590-c81b2e2cbb2c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:38.162477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283b253-887e-4b0a-8f90-5b3f85d2dbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:38.163083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b253-887e-4b0a-8f90-5b3f85d2dbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:38.163355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b253-887e-4b0a-8f90-5b3f85d2dbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:38.163629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b253-887e-4b0a-8f90-5b3f85d2dbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:38.173063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283b253-887e-4b0a-8f90-5b3f85d2dbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:38.174555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283b253-887e-4b0a-8f90-5b3f85d2dbd6 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:38.185234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 756/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:41.696084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e6ae-0d59-4353-bac0-117e8ccf44ee None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:41.696607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e6ae-0d59-4353-bac0-117e8ccf44ee None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:41.696607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e6ae-0d59-4353-bac0-117e8ccf44ee None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:41.697964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:30:41.703308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fbaec-8ce3-4146-bf98-3abff0f03a21 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:41.703731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fbaec-8ce3-4146-bf98-3abff0f03a21 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:41.703842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fbaec-8ce3-4146-bf98-3abff0f03a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:41.712367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=81 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:41.725677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b48fbaec-8ce3-4146-bf98-3abff0f03a21 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:41.731377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b48fbaec-8ce3-4146-bf98-3abff0f03a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:41.735365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:42.047931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e6ae-0d59-4353-bac0-117e8ccf44ee None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:42.049698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9288e6ae-0d59-4353-bac0-117e8ccf44ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:42.118543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:42.132263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e6ae-0d59-4353-bac0-117e8ccf44ee None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:42.135889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6672a-4300-4eaf-b6f3-bfb845d890f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:42.136486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6672a-4300-4eaf-b6f3-bfb845d890f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:42.136802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6672a-4300-4eaf-b6f3-bfb845d890f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:42.137020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6672a-4300-4eaf-b6f3-bfb845d890f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:42.148144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6672a-4300-4eaf-b6f3-bfb845d890f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:42.150064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6672a-4300-4eaf-b6f3-bfb845d890f9 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:42.160483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 779/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:44.219966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139c20-2089-46ee-8050-daf5cccf2a62 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:44.220480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139c20-2089-46ee-8050-daf5cccf2a62 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:44.220480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139c20-2089-46ee-8050-daf5cccf2a62 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:44.221787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:30:44.227623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869cfb4-9b89-40c8-87d1-ab40ac68d2e7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:44.228022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869cfb4-9b89-40c8-87d1-ab40ac68d2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:44.228178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869cfb4-9b89-40c8-87d1-ab40ac68d2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:44.248241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7869cfb4-9b89-40c8-87d1-ab40ac68d2e7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:44.249446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7869cfb4-9b89-40c8-87d1-ab40ac68d2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:44.258268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:44.565701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139c20-2089-46ee-8050-daf5cccf2a62 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:44.567443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-97139c20-2089-46ee-8050-daf5cccf2a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:44.580712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139c20-2089-46ee-8050-daf5cccf2a62 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:44.584840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a8354-8657-4e42-afa3-1d969a8c0afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:44.585430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a8354-8657-4e42-afa3-1d969a8c0afe service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:44.585709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a8354-8657-4e42-afa3-1d969a8c0afe service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:44.585980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a8354-8657-4e42-afa3-1d969a8c0afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:44.595031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a8354-8657-4e42-afa3-1d969a8c0afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:44.596452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a8354-8657-4e42-afa3-1d969a8c0afe service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:44.606255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 759/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:54.243599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=11 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:54.602178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=8 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:07.673986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b559bbd-b4f6-4801-8d94-d41cb3d8d469 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.674990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b559bbd-b4f6-4801-8d94-d41cb3d8d469 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.678953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b559bbd-b4f6-4801-8d94-d41cb3d8d469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.698866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2b559bbd-b4f6-4801-8d94-d41cb3d8d469 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:07.700080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2b559bbd-b4f6-4801-8d94-d41cb3d8d469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:07.710039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 781/1540] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:07.720138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d4d19-8549-46f6-ba58-8805f9752150 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.720716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d4d19-8549-46f6-ba58-8805f9752150 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.720716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d4d19-8549-46f6-ba58-8805f9752150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.749569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-743d4d19-8549-46f6-ba58-8805f9752150 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:07.750881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-743d4d19-8549-46f6-ba58-8805f9752150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:07.767632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 760/1541] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:07.778034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b559bbd-b4f6-4801-8d94-d41cb3d8d469 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:07.791041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ef873-8865-486e-a6de-1165c4ce6d1c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.791520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ef873-8865-486e-a6de-1165c4ce6d1c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.791869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ef873-8865-486e-a6de-1165c4ce6d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.791990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b559bbd-b4f6-4801-8d94-d41cb3d8d469 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:07.800658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:07.801377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.801655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.802013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.802727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:07.803013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:07.804760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:07.828819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 782/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:07.830141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-925ef873-8865-486e-a6de-1165c4ce6d1c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:07.831389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-925ef873-8865-486e-a6de-1165c4ce6d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:07.838209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:07.846154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 761/1543] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:07.855303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c8a5d-c7eb-4965-91f7-cdd4bc831917 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.855648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c8a5d-c7eb-4965-91f7-cdd4bc831917 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.855844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c8a5d-c7eb-4965-91f7-cdd4bc831917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.857008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28f5b-1f2e-42f2-aac4-6e9e3fa085b3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:07.876205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:07.876832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.877112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.877383 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.878030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:07.883303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:32:07.883303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:32:07.883303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:32:07.885176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:07.895709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cc5c8a5d-c7eb-4965-91f7-cdd4bc831917 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:07.896930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cc5c8a5d-c7eb-4965-91f7-cdd4bc831917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:07.915084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 762/1544] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:07.916783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b64dd-3134-46f2-9a8a-1db75afc92c0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.917063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b64dd-3134-46f2-9a8a-1db75afc92c0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.917337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b64dd-3134-46f2-9a8a-1db75afc92c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.944789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-368b64dd-3134-46f2-9a8a-1db75afc92c0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:07.946038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-368b64dd-3134-46f2-9a8a-1db75afc92c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:07.956202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 763/1545] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:07.956202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a3a6-b296-4994-a194-0d07b92f66aa None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.956202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a3a6-b296-4994-a194-0d07b92f66aa None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.956486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a3a6-b296-4994-a194-0d07b92f66aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:07.977030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1ca8a3a6-b296-4994-a194-0d07b92f66aa None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:07.978570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1ca8a3a6-b296-4994-a194-0d07b92f66aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:07.987385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 764/1546] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:07.988735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa110f4-9755-4f0d-abf0-6927fec5de26 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:07.989020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa110f4-9755-4f0d-abf0-6927fec5de26 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:07.989284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa110f4-9755-4f0d-abf0-6927fec5de26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.010040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-baa110f4-9755-4f0d-abf0-6927fec5de26 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.011258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-baa110f4-9755-4f0d-abf0-6927fec5de26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.019020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 765/1547] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.020243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3a287-cbd7-4458-a828-1ec9c059ddf2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.020611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3a287-cbd7-4458-a828-1ec9c059ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.020833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3a287-cbd7-4458-a828-1ec9c059ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.030428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 783/1548] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:07 2022] POST /identity/v3/projects => generated 389 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:08.031814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.046166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-94c3a287-cbd7-4458-a828-1ec9c059ddf2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.047410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-94c3a287-cbd7-4458-a828-1ec9c059ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.104540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a89ee8-7642-4525-93e5-b27aa15f4c47 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.107984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 766/1549] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.109228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cd154-e860-4bd5-b9a0-10384c382fae None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.109515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cd154-e860-4bd5-b9a0-10384c382fae None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.109789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cd154-e860-4bd5-b9a0-10384c382fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.111720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0786d-aef8-4f67-84ab-1883a447f435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.112437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0786d-aef8-4f67-84ab-1883a447f435 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.112817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0786d-aef8-4f67-84ab-1883a447f435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.113088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0786d-aef8-4f67-84ab-1883a447f435 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.113954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0786d-aef8-4f67-84ab-1883a447f435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:08.114419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0786d-aef8-4f67-84ab-1883a447f435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.116155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0786d-aef8-4f67-84ab-1883a447f435 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.137055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 784/1550] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:08.137055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cbbe5-c8d4-47ea-a280-3e3a59b55f2f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.137055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cbbe5-c8d4-47ea-a280-3e3a59b55f2f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.137642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-881cd154-e860-4bd5-b9a0-10384c382fae None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.137854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cbbe5-c8d4-47ea-a280-3e3a59b55f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.138098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-881cd154-e860-4bd5-b9a0-10384c382fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.140477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 767/1551] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.141693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3796bac-5a28-4fc8-bc8b-412b43ea4048 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.141972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3796bac-5a28-4fc8-bc8b-412b43ea4048 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.142232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3796bac-5a28-4fc8-bc8b-412b43ea4048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.150733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c78cbbe5-c8d4-47ea-a280-3e3a59b55f2f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.151954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c78cbbe5-c8d4-47ea-a280-3e3a59b55f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.159989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 785/1552] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.161151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132071c-bdc4-42d2-80b8-3065841bf281 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.161430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132071c-bdc4-42d2-80b8-3065841bf281 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.161700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132071c-bdc4-42d2-80b8-3065841bf281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.161928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d3796bac-5a28-4fc8-bc8b-412b43ea4048 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.163127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d3796bac-5a28-4fc8-bc8b-412b43ea4048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.171776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 768/1553] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.172521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3796bac-5a28-4fc8-bc8b-412b43ea4048 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.182534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3796bac-5a28-4fc8-bc8b-412b43ea4048 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.185065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4132071c-bdc4-42d2-80b8-3065841bf281 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.186251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4132071c-bdc4-42d2-80b8-3065841bf281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.187052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6bacc-b6f4-44d5-b032-27e90a9144f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.187703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6bacc-b6f4-44d5-b032-27e90a9144f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.188047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6bacc-b6f4-44d5-b032-27e90a9144f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.188316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6bacc-b6f4-44d5-b032-27e90a9144f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.189082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6bacc-b6f4-44d5-b032-27e90a9144f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.190090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-28a6bacc-b6f4-44d5-b032-27e90a9144f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:32:08.190090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Aug 08 02:32:08.190090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: Aug 08 02:32:08.191316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6bacc-b6f4-44d5-b032-27e90a9144f0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.194295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 786/1554] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.195545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39025c82-e44d-4929-9407-868d9329a57d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.195824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39025c82-e44d-4929-9407-868d9329a57d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.196121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39025c82-e44d-4929-9407-868d9329a57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.216609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-39025c82-e44d-4929-9407-868d9329a57d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.217791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-39025c82-e44d-4929-9407-868d9329a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.225783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 787/1555] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.227117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5caa3-4eaa-493c-8398-acb3f9cc00cb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.227922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5caa3-4eaa-493c-8398-acb3f9cc00cb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.228392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5caa3-4eaa-493c-8398-acb3f9cc00cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.248049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-91a5caa3-4eaa-493c-8398-acb3f9cc00cb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.249211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-91a5caa3-4eaa-493c-8398-acb3f9cc00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.256897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 788/1556] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.257753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5caa3-4eaa-493c-8398-acb3f9cc00cb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.267200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5caa3-4eaa-493c-8398-acb3f9cc00cb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.271682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969940a-dad0-4113-b31e-c24ffdaa91d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.272238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969940a-dad0-4113-b31e-c24ffdaa91d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.272509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969940a-dad0-4113-b31e-c24ffdaa91d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.272775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969940a-dad0-4113-b31e-c24ffdaa91d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.273370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969940a-dad0-4113-b31e-c24ffdaa91d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.275649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969940a-dad0-4113-b31e-c24ffdaa91d3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.296361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 769/1557] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:08.297648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ccde-bc50-4406-ba6e-cf5aa38e9576 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.297953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ccde-bc50-4406-ba6e-cf5aa38e9576 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.298222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ccde-bc50-4406-ba6e-cf5aa38e9576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.317528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4845ccde-bc50-4406-ba6e-cf5aa38e9576 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.318772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4845ccde-bc50-4406-ba6e-cf5aa38e9576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.326525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 770/1558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.327262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845ccde-bc50-4406-ba6e-cf5aa38e9576 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.336165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845ccde-bc50-4406-ba6e-cf5aa38e9576 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.341242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.341795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.342066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.342332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.342979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:08.343283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.344268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:32:08.344268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Aug 08 02:32:08.344268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: Aug 08 02:32:08.345223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.357397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 771/1559] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:08.358312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.367475 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2531487-bde4-416a-a8f9-ca61611a5835 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.372976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.373534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.373809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.374076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.374760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:08.375015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.376595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.388578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 772/1560] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:08.389506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.395328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 789/1561] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:08.397136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be394e-5ca7-48cd-9da4-0127c4de2713 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.397456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be394e-5ca7-48cd-9da4-0127c4de2713 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.397729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be394e-5ca7-48cd-9da4-0127c4de2713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.403870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ab2d2-1b40-46dc-be6e-58e98b3d68c3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.418646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-73be394e-5ca7-48cd-9da4-0127c4de2713 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:08.419829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-73be394e-5ca7-48cd-9da4-0127c4de2713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.456208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:08.462795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.463394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.463670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.463942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.464816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:08.464947 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.466472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.467763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:08.478449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 773/1562] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:08.479279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.479896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 790/1563] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:08.480609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be394e-5ca7-48cd-9da4-0127c4de2713 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.491066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be394e-5ca7-48cd-9da4-0127c4de2713 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.495708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dface3-f80a-4263-a30b-b61970d76f10 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.496276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.496856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.497132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.497403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.498040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.499657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.500389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.501036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.501336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.501613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.502242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:08.502572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.504208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.513817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 791/1564] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:08.513817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.523490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 774/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:08.523490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.523770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbded84-629e-4284-9855-7ccb895f0ff6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.527148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.527732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f2290-14f4-4cce-9895-88410b9a3083 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.527845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.528003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.528269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.528903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:08.529292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.530834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.533202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.533785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.534064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.534337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.535037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.536921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.543432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 792/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:08.544206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.555168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47da36-44a6-419d-9225-ee8b060e9611 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.560546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.564025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.564025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.564025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.564025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.564389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.611598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 775/1567] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:08.612548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.641458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 793/1568] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:08.641458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.677233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b11748-ac7a-4c44-9f8e-19815ffe3723 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.708109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a55378-65a8-47c0-91dc-2d83cd5ec6f4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.713590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.714262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.714570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.714872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.715508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:08.715810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.717400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.729615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 794/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:08.730416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.739565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2eab2a-2ba9-4faa-bc68-6d4586993ee6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.742408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.743162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.743453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.743735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.744302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.744429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.744865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.745138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.745419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.745999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.746403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.747873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.850734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 776/1570] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:08.851554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.861196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940de9df-329e-4eb9-aa22-93d892d2f9dc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.865860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.866471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.866753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.867025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.867647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.869283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.879352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 777/1571] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:08.880031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 795/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:08.882216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.882337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.891940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c4f6f-7b0f-489d-9592-8200babe812a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.932989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:08.946802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190bb2d-1919-40e5-99f3-90fddb2ea5e1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.956735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.956735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.956735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.956735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.957143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.957259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.957370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.957370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.957370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.957370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.957735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.957822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.143224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 778/1573] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/projects => generated 357 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.144063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.156403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 796/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:08 2022] POST /identity/v3/projects => generated 373 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.157081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.168295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:09.207402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6dc58a-2752-49ec-b9ec-3d29ee5cfefd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.221967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd83288-0b1c-47f8-b730-0462259c3559 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.226815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.227631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.227916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.228192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.228820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.230780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.263261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.263857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.264131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.264396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.264999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.266996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.301882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 797/1575] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.302704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.313225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83522fc-834b-4324-9501-e911a6fa1843 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.317851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.318462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.318742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.319012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.319622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.322233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.333656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 779/1576] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.333656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.344307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde031-dd37-4728-a653-17857477ac01 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.346142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.346775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.347073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.347344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.347969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.349543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.359701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 780/1577] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.360460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.370318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338937d-0033-4a75-880a-6f2d94c1feef admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.374873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.375473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.375748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.376012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] PATH_INFO: `/v3/projects/dddb64c31c0a4db2a184543c435c7bc0/users/7fd9eb21aa634650a7022b7b4ca145fd/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.391044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dddb64c31c0a4db2a184543c435c7bc0, user_id=7fd9eb21aa634650a7022b7b4ca145fd, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.393694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.431856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 798/1578] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.431856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.484400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 781/1579] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:09 2022] PUT /identity/v3/projects/dddb64c31c0a4db2a184543c435c7bc0/users/7fd9eb21aa634650a7022b7b4ca145fd/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:09.485210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.490599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35e1b-77e4-435b-aabf-bea5138b1509 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.517390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:09.548311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b3c03-18a5-4789-adc0-6693ac20e073 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.548967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.549657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.549928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.550193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.551001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.552898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.553059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.553682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.553955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.554220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.554862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.557456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.654237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 799/1580] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.654451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 782/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/users => generated 279 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.654585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.654682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.662164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63fe88-4408-4217-b274-cb6723ca481f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.667635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.667904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4eb32-3e78-4aba-b64c-074291bec62c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.668287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.668560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.668927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.669904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.671177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.673083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0acaf-5588-4ee7-9be9-74c5e8da6335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.674043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0acaf-5588-4ee7-9be9-74c5e8da6335 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.674324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0acaf-5588-4ee7-9be9-74c5e8da6335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.674627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0acaf-5588-4ee7-9be9-74c5e8da6335 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.675253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0acaf-5588-4ee7-9be9-74c5e8da6335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.677657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0acaf-5588-4ee7-9be9-74c5e8da6335 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.682053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 783/1582] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.682888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.688720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:09.693059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c974bc-2a25-4e8e-98a1-6a3dcad420c7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.697915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.698533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.698815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.699085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.699743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.701418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.712191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 784/1583] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.713079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.722413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be1260-df54-47d5-a53f-1222de2900c1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.727084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.728110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.728477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.728751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.729354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.730933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.740437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 785/1584] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.741278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.751143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06039b-85b2-4dee-964c-d3779ec14a6f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.755906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.756456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.756733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.757005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.757599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.759523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.772365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 800/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.772850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205692-483a-4e00-ad90-8c955cac77cf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.772850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205692-483a-4e00-ad90-8c955cac77cf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.775680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9205692-483a-4e00-ad90-8c955cac77cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.792997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e9205692-483a-4e00-ad90-8c955cac77cf None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:09.794181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e9205692-483a-4e00-ad90-8c955cac77cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:09.801731 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 801/1586] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:09.802504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9205692-483a-4e00-ad90-8c955cac77cf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.812510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9205692-483a-4e00-ad90-8c955cac77cf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.817238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.817842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.818116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.818413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.819045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.820603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.830632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 802/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.831521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.841366 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdabc3-8ced-4ff1-ba2c-2becd7f40ed7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.846284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.850229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.850229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.850229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.850229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.850229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.860266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 803/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.861200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.871812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f03d-592c-487f-a1a4-33a8fb86125a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.876649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.877252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.877525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.877795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.878493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.881983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 786/1589] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/users => generated 282 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.881983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.882201 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.890273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51c7c8-dd20-48b9-8ec7-201657f32812 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.892281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 804/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.892281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.895677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.896278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.896675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.896954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.897598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.899893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.902427 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b719fe-7472-4aa3-836b-accac5ec101b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.907151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.907720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.907994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.908266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] PATH_INFO: `/v3/projects/ce575522168344caacf13cfc7779e25b/users/d725e4cb658b43438c1f22a80b1dc5ad/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.910840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 787/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.911629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.924089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d35576-4397-432d-bdce-88fc63887ce3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.928925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.929527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.929655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce575522168344caacf13cfc7779e25b, user_id=d725e4cb658b43438c1f22a80b1dc5ad, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.929809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.930075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.930748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:32:09.930748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:32:09.930748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:32:09.931304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.932638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.932746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.942870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 788/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.943680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.000142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d731a7-6a88-40ec-adfa-bc79a37e5ae9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.003895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 805/1593] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:09 2022] PUT /identity/v3/projects/ce575522168344caacf13cfc7779e25b/users/d725e4cb658b43438c1f22a80b1dc5ad/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.007847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.049209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.049575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.052970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.052970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.063470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:10.068421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34be25-1845-46af-9798-3828b7f836a2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.073276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.073872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.074145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.074445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.075078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.077235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.087956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 806/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:10.088742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.098675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656ba0e-8647-4f9c-9ede-be457b67ff3f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.104107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.104786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.105142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.105448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.106091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.107707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.116289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.117820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.117946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 807/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:10.118907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.127955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 789/1596] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:10.128658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc35d2-7d52-4080-8ffe-86d1460b0514 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.128811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.133691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.134170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.134851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.134851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.135380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.136957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.138549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6cee7-3368-4515-963f-2379d1cd9470 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.143266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f6c99-9440-4b9f-b3c0-ac0187908edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.143922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f6c99-9440-4b9f-b3c0-ac0187908edd admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.144195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f6c99-9440-4b9f-b3c0-ac0187908edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.144465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f6c99-9440-4b9f-b3c0-ac0187908edd admin admin] PATH_INFO: `/v3/projects/f1f194384993444da5c7b7f30e42746d/users/f6fb96d9442147e0a2201fe890d6f955/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.150669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 808/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:10.154300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.161647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f6c99-9440-4b9f-b3c0-ac0187908edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f194384993444da5c7b7f30e42746d, user_id=f6fb96d9442147e0a2201fe890d6f955, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.163485 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f6c99-9440-4b9f-b3c0-ac0187908edd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.164273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85edc7-aab2-46d4-96a0-b34ecf6268cd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.169986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.170658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.170938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.171210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] PATH_INFO: `/v3/projects/8dd587c816564259a98f16356ea478d0/users/3a3b08b59b9c4e8ebe92821fe3de149e/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.185455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd587c816564259a98f16356ea478d0, user_id=3a3b08b59b9c4e8ebe92821fe3de149e, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.188134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.274970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 790/1598] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/f1f194384993444da5c7b7f30e42746d/users/f6fb96d9442147e0a2201fe890d6f955/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.278780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 809/1599] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/8dd587c816564259a98f16356ea478d0/users/3a3b08b59b9c4e8ebe92821fe3de149e/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.278780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.280569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bcf4c-8154-41ac-9706-c979d102b26e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.280886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bcf4c-8154-41ac-9706-c979d102b26e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.281147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bcf4c-8154-41ac-9706-c979d102b26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.303831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e75bcf4c-8154-41ac-9706-c979d102b26e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:10.305086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e75bcf4c-8154-41ac-9706-c979d102b26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:10.335356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:10.356788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533185fe-bdf5-4d83-a225-7116529fa096 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.361607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.362229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.362548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.362822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.366821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.369242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.372048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 791/1600] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:10.373195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bcf4c-8154-41ac-9706-c979d102b26e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.386004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 810/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:10.386188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bcf4c-8154-41ac-9706-c979d102b26e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.386313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.387792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.388568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.388875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.389141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] PATH_INFO: `/v3/projects/d928ecfce43944ee80fa940e6f363a22/users/81bf2a5a130d4204893204af69abb74c/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.395663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596335f-247f-4d46-a480-3b7a83524a39 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.401284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002a69e-6af0-45ea-a0b3-9fdd0d4b6911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.402049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002a69e-6af0-45ea-a0b3-9fdd0d4b6911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.402331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002a69e-6af0-45ea-a0b3-9fdd0d4b6911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.402620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002a69e-6af0-45ea-a0b3-9fdd0d4b6911 admin admin] PATH_INFO: `/v3/projects/f23d6ea67ba84884b28ecad806b75589/users/5f59c6c53a67481e83d673251d469e13/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.404519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d928ecfce43944ee80fa940e6f363a22, user_id=81bf2a5a130d4204893204af69abb74c, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.407187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.417775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002a69e-6af0-45ea-a0b3-9fdd0d4b6911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23d6ea67ba84884b28ecad806b75589, user_id=5f59c6c53a67481e83d673251d469e13, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.420385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002a69e-6af0-45ea-a0b3-9fdd0d4b6911 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.497017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 792/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/d928ecfce43944ee80fa940e6f363a22/users/81bf2a5a130d4204893204af69abb74c/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.497763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.533033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 811/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/f23d6ea67ba84884b28ecad806b75589/users/5f59c6c53a67481e83d673251d469e13/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.534472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c87bc-357e-4065-96da-285ae3fbae30 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.534914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c87bc-357e-4065-96da-285ae3fbae30 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.535246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c87bc-357e-4065-96da-285ae3fbae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.557210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-355c87bc-357e-4065-96da-285ae3fbae30 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:10.558536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-355c87bc-357e-4065-96da-285ae3fbae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:10.573921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf690ef-8e24-40bb-a017-d24027225126 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.581857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:10.620607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 812/1604] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:10.621855 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426598d-2af3-4999-a9c6-08748ec5ff64 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.622132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426598d-2af3-4999-a9c6-08748ec5ff64 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.622484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426598d-2af3-4999-a9c6-08748ec5ff64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.645057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.645678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.645961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.646235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.647038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.649707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.651087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2426598d-2af3-4999-a9c6-08748ec5ff64 None None] MFA Rules not processed for user `3a3b08b59b9c4e8ebe92821fe3de149e`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:10.652325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2426598d-2af3-4999-a9c6-08748ec5ff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:10.659955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 793/1605] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:10.663913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.724818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 813/1606] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:10.725446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426598d-2af3-4999-a9c6-08748ec5ff64 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.727121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988312a-9252-4c6a-b5e0-4f12245aa34c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.732205 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aeddd0-8d4f-4360-92dd-611873f570be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.732843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeddd0-8d4f-4360-92dd-611873f570be service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.733120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeddd0-8d4f-4360-92dd-611873f570be service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.733392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeddd0-8d4f-4360-92dd-611873f570be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.739643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426598d-2af3-4999-a9c6-08748ec5ff64 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.743828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aeddd0-8d4f-4360-92dd-611873f570be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.748085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b36eef-7d66-4d57-9aa9-2e728556b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.748258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aeddd0-8d4f-4360-92dd-611873f570be service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.748358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b36eef-7d66-4d57-9aa9-2e728556b9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.748358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b36eef-7d66-4d57-9aa9-2e728556b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.748358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b36eef-7d66-4d57-9aa9-2e728556b9b4 admin admin] PATH_INFO: `/v3/projects/b86941379d2b4d7f81e0dd09a0494692/users/b532fee72b104c8094a63a738a37bf09/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.755752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 794/1607] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:10.757012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87e624-c713-45f0-af57-bf3ab47c2430 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.757726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87e624-c713-45f0-af57-bf3ab47c2430 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.757726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87e624-c713-45f0-af57-bf3ab47c2430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.778317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b36eef-7d66-4d57-9aa9-2e728556b9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86941379d2b4d7f81e0dd09a0494692, user_id=b532fee72b104c8094a63a738a37bf09, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.782522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b36eef-7d66-4d57-9aa9-2e728556b9b4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.787040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-de87e624-c713-45f0-af57-bf3ab47c2430 None None] MFA Rules not processed for user `81bf2a5a130d4204893204af69abb74c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:10.789397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-de87e624-c713-45f0-af57-bf3ab47c2430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:10.851031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 814/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/b86941379d2b4d7f81e0dd09a0494692/users/b532fee72b104c8094a63a738a37bf09/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.852768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2664f-b4e7-43f5-a72c-16c4ffbc23a5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.853213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2664f-b4e7-43f5-a72c-16c4ffbc23a5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.853638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2664f-b4e7-43f5-a72c-16c4ffbc23a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.873222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 795/1609] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:10.873222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87e624-c713-45f0-af57-bf3ab47c2430 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.885490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87e624-c713-45f0-af57-bf3ab47c2430 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.890584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6dd2664f-b4e7-43f5-a72c-16c4ffbc23a5 None None] MFA Rules not processed for user `5f59c6c53a67481e83d673251d469e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:10.891943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6dd2664f-b4e7-43f5-a72c-16c4ffbc23a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:10.897295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:10.935483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:10.951933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.951933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.954022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.954557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.979486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 815/1610] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:10.983923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2664f-b4e7-43f5-a72c-16c4ffbc23a5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.027991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.029556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.040783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 796/1611] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:11.041663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.051913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85299fbe-02eb-4630-bd78-909fcef53425 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.057615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618bc87-cad6-4a76-a35c-f685bb6b4e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.058224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618bc87-cad6-4a76-a35c-f685bb6b4e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.058539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618bc87-cad6-4a76-a35c-f685bb6b4e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.058802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618bc87-cad6-4a76-a35c-f685bb6b4e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.059422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618bc87-cad6-4a76-a35c-f685bb6b4e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.061476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618bc87-cad6-4a76-a35c-f685bb6b4e1f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.079117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2664f-b4e7-43f5-a72c-16c4ffbc23a5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.084369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.085029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.085309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.085581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] PATH_INFO: `/v3/projects/84bd2e47b642474f8f51f028747a8e77/users/004ac2aedb8648139be9c3f04653aa16/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.101878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=84bd2e47b642474f8f51f028747a8e77, user_id=004ac2aedb8648139be9c3f04653aa16, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.104723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.158651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 797/1612] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:11.158651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a65dc-b971-487f-a3c1-aed934e54256 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.158651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a65dc-b971-487f-a3c1-aed934e54256 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.158651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a65dc-b971-487f-a3c1-aed934e54256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.177175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 816/1613] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/84bd2e47b642474f8f51f028747a8e77/users/004ac2aedb8648139be9c3f04653aa16/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:11.178315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.198875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-995a65dc-b971-487f-a3c1-aed934e54256 None None] MFA Rules not processed for user `b532fee72b104c8094a63a738a37bf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:11.203787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-995a65dc-b971-487f-a3c1-aed934e54256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:11.252484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e40b8-e6a5-4fa1-a78f-b1df337c26ae admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.257887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.259073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.259506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.259987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.260610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.262724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.278478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 798/1614] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:11.279470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a65dc-b971-487f-a3c1-aed934e54256 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.291339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a65dc-b971-487f-a3c1-aed934e54256 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.296167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5290283-f87a-46e8-b69d-ccf553c79544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.296958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5290283-f87a-46e8-b69d-ccf553c79544 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.297176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5290283-f87a-46e8-b69d-ccf553c79544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.297378 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5290283-f87a-46e8-b69d-ccf553c79544 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.297997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5290283-f87a-46e8-b69d-ccf553c79544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.300382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5290283-f87a-46e8-b69d-ccf553c79544 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.373220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 817/1615] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/projects => generated 355 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:11.374469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.399945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:11.406675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 799/1616] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:11.408074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5024-a322-46f1-8f2c-b9d788cbecdb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.408624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5024-a322-46f1-8f2c-b9d788cbecdb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.409106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5024-a322-46f1-8f2c-b9d788cbecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.442250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-470b5024-a322-46f1-8f2c-b9d788cbecdb None None] MFA Rules not processed for user `004ac2aedb8648139be9c3f04653aa16`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:11.443643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-470b5024-a322-46f1-8f2c-b9d788cbecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:11.452117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ad560-8e2a-4abe-b1be-b617b020678f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.470923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:11.511571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.512364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.512716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.513254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.513890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.516933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.519575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 800/1617] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:11.525080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd633d-ac4a-4fbc-90a0-819d5d3fffe3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.525367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd633d-ac4a-4fbc-90a0-819d5d3fffe3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.526202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd633d-ac4a-4fbc-90a0-819d5d3fffe3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.527215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 801/1618] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 02:32:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:11.530915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd633d-ac4a-4fbc-90a0-819d5d3fffe3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.544299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd633d-ac4a-4fbc-90a0-819d5d3fffe3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.549255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.549886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.550161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.550469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.551119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.553072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.591552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 818/1619] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:11.592504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.605316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfcc7f-66b6-4c67-8014-f9f093dab5b6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.612593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db170e0d-d993-4d9f-b2c0-9b37f43813e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.613435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db170e0d-d993-4d9f-b2c0-9b37f43813e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.613786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db170e0d-d993-4d9f-b2c0-9b37f43813e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.614147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db170e0d-d993-4d9f-b2c0-9b37f43813e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.615000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db170e0d-d993-4d9f-b2c0-9b37f43813e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.617362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db170e0d-d993-4d9f-b2c0-9b37f43813e4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.653304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 802/1620] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:11.654037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.732284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 819/1621] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:11.732284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c139c-abfb-4536-99b0-5b3b7fc1217d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.732284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c139c-abfb-4536-99b0-5b3b7fc1217d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.732284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c139c-abfb-4536-99b0-5b3b7fc1217d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.756223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a90c5b-eca0-4304-ace9-a96d2d7ee609 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.765651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.766789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.767068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.767545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.778164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.779887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.784610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-236c139c-abfb-4536-99b0-5b3b7fc1217d None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:11.785864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-236c139c-abfb-4536-99b0-5b3b7fc1217d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:11.789820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 803/1622] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:11.790928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.802017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866d6f-2485-4e9f-a11b-a60c7abfca0a service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.826077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.826813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.827183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.827530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.828239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.829940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.859224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 804/1623] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:11.860401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.897042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74584e70-43cc-42fa-883b-7d5c8edeb314 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.900109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 820/1624] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:11.901451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c139c-abfb-4536-99b0-5b3b7fc1217d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.911049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.911672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.911953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.912227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.912879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.917056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c139c-abfb-4536-99b0-5b3b7fc1217d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.917210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.919617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.920409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.920780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.921601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.922074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.924160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.940105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 805/1625] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:11.942483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 821/1626] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:11.944536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.945306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.962956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7fb4-1d89-4342-9b5d-3f9a578c3c14 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.966254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfde2e-649f-47a7-aa69-c5c01953ec49 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.970061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.970939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.971164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.971439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.983872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.984848 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.985114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.985395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] PATH_INFO: `/v3/projects/4dbadb049eff4805bc99ed7515eb4088/users/194d0754979149fe937f8045097df030/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.017802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dbadb049eff4805bc99ed7515eb4088, user_id=194d0754979149fe937f8045097df030, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.022704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.068659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:12.072695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.075119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.098507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 822/1627] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:12.098507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.108370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41a75a-1f8c-4632-9b0a-a2aa0440de1f service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.114216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.115361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.115759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.116633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] PATH_INFO: `/v3/projects/174eb651d83d46a6922d3a22b7ed207a/users/abbee575ca9d42c6a1e7d3f06430c2dd/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.127854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 806/1628] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:11 2022] PUT /identity/v3/projects/4dbadb049eff4805bc99ed7515eb4088/users/194d0754979149fe937f8045097df030/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:12.128747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.142725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=174eb651d83d46a6922d3a22b7ed207a, user_id=abbee575ca9d42c6a1e7d3f06430c2dd, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.145917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.248150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 823/1629] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:12 2022] PUT /identity/v3/projects/174eb651d83d46a6922d3a22b7ed207a/users/abbee575ca9d42c6a1e7d3f06430c2dd/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:12.249685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.257588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79209f8c-80a8-4e67-bfc3-0552fa8fc43f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.278894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:12.359853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebeafc-afa1-4831-9513-8e6b76ec5fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.360610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebeafc-afa1-4831-9513-8e6b76ec5fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.360994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebeafc-afa1-4831-9513-8e6b76ec5fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.361318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebeafc-afa1-4831-9513-8e6b76ec5fd7 admin admin] PATH_INFO: `/v3/projects/eb8b786327504e7da407c1137bd1e5d7/users/cd00f6183cef43819c90d7fbd398da80/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.376333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50e88c-d89b-4a2a-994d-3ece978cb757 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.385948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebeafc-afa1-4831-9513-8e6b76ec5fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8b786327504e7da407c1137bd1e5d7, user_id=cd00f6183cef43819c90d7fbd398da80, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.390614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebeafc-afa1-4831-9513-8e6b76ec5fd7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.392894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.393737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.398645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.399177 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.400013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.401804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.428430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 824/1630] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:12.433147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.453708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e03018-e747-4b2f-b573-ed6e80c01f08 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.467442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.468611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.468886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.469161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.469822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.471485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.492116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 825/1631] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:12.493757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.505194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab798be0-34aa-4de2-bac6-470864553890 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.512764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.513533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.517275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.517275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] PATH_INFO: `/v3/projects/4dbadb049eff4805bc99ed7515eb4088/users/194d0754979149fe937f8045097df030/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.517275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dbadb049eff4805bc99ed7515eb4088, user_id=194d0754979149fe937f8045097df030, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.517638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 807/1632] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:12 2022] PUT /identity/v3/projects/eb8b786327504e7da407c1137bd1e5d7/users/cd00f6183cef43819c90d7fbd398da80/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:12.519514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d9fa-f339-4cb2-aea6-2aa9d024ea15 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.519812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d9fa-f339-4cb2-aea6-2aa9d024ea15 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.520166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d9fa-f339-4cb2-aea6-2aa9d024ea15 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.521650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 808/1633] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 02:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:12.522816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.523022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d9fa-f339-4cb2-aea6-2aa9d024ea15 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.542870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 826/1634] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:12 2022] PUT /identity/v3/projects/4dbadb049eff4805bc99ed7515eb4088/users/194d0754979149fe937f8045097df030/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:12.543930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.623134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bcf90-a22d-4987-8dfa-e9fd17d49a34 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.628803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2b762-8e84-4c50-9791-7da6510295ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.629669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2b762-8e84-4c50-9791-7da6510295ae admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.630085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2b762-8e84-4c50-9791-7da6510295ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.630515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2b762-8e84-4c50-9791-7da6510295ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.631526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2b762-8e84-4c50-9791-7da6510295ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.633389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2b762-8e84-4c50-9791-7da6510295ae admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.637738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d9fa-f339-4cb2-aea6-2aa9d024ea15 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.646789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 827/1635] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:12.649123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20515d-bb16-4cb8-8cf8-8c124a06cca4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.650221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20515d-bb16-4cb8-8cf8-8c124a06cca4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.650596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3620c5-cf66-461a-a585-84d32d35c0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.650753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20515d-bb16-4cb8-8cf8-8c124a06cca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.651342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3620c5-cf66-461a-a585-84d32d35c0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.651620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3620c5-cf66-461a-a585-84d32d35c0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.651895 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3620c5-cf66-461a-a585-84d32d35c0ad admin admin] PATH_INFO: `/v3/projects/174eb651d83d46a6922d3a22b7ed207a/users/abbee575ca9d42c6a1e7d3f06430c2dd/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.658267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3620c5-cf66-461a-a585-84d32d35c0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=174eb651d83d46a6922d3a22b7ed207a, user_id=abbee575ca9d42c6a1e7d3f06430c2dd, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.665147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3620c5-cf66-461a-a585-84d32d35c0ad admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.677894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9c20515d-bb16-4cb8-8cf8-8c124a06cca4 None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:12.679646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9c20515d-bb16-4cb8-8cf8-8c124a06cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:12.704711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 809/1636] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:12 2022] PUT /identity/v3/projects/174eb651d83d46a6922d3a22b7ed207a/users/abbee575ca9d42c6a1e7d3f06430c2dd/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:12.709240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c107333-9ad9-4d12-9e4b-29e960686cd7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.709240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c107333-9ad9-4d12-9e4b-29e960686cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.709240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c107333-9ad9-4d12-9e4b-29e960686cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.744975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2c107333-9ad9-4d12-9e4b-29e960686cd7 None None] MFA Rules not processed for user `194d0754979149fe937f8045097df030`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:12.746824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2c107333-9ad9-4d12-9e4b-29e960686cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:12.783522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 828/1637] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:12.784630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20515d-bb16-4cb8-8cf8-8c124a06cca4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.826823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 810/1638] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:12.828283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb865cb-12ac-4f97-9af5-51e610ed2e34 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.828630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb865cb-12ac-4f97-9af5-51e610ed2e34 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.828963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb865cb-12ac-4f97-9af5-51e610ed2e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.879088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20515d-bb16-4cb8-8cf8-8c124a06cca4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.906627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5cb865cb-12ac-4f97-9af5-51e610ed2e34 None None] MFA Rules not processed for user `abbee575ca9d42c6a1e7d3f06430c2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:12.916464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5cb865cb-12ac-4f97-9af5-51e610ed2e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:12.924181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.924997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.933116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.933116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] PATH_INFO: `/v3/projects/eb8b786327504e7da407c1137bd1e5d7/users/cd00f6183cef43819c90d7fbd398da80/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.933116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8b786327504e7da407c1137bd1e5d7, user_id=cd00f6183cef43819c90d7fbd398da80, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.933116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.944537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 829/1639] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:12 2022] PUT /identity/v3/projects/eb8b786327504e7da407c1137bd1e5d7/users/cd00f6183cef43819c90d7fbd398da80/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:12.945462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:13.051132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 811/1640] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:13.052706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8e10d-8ab8-44f3-a3af-df7039f1bb2c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:13.053738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:13.060440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8e10d-8ab8-44f3-a3af-df7039f1bb2c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:13.060440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8e10d-8ab8-44f3-a3af-df7039f1bb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:13.062524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb599f-e5e0-4004-b894-3d68f1739058 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:13.080731 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:13.081363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:13.081667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:13.081942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:13.124187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a8b8e10d-8ab8-44f3-a3af-df7039f1bb2c None None] MFA Rules not processed for user `cd00f6183cef43819c90d7fbd398da80`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:13.125427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a8b8e10d-8ab8-44f3-a3af-df7039f1bb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:13.131213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:13.329880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:13.329880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:13.345997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 830/1641] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:13.351523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:13.371061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62548744-94d8-4c23-bf2f-946d533e2348 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:13.386951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:13.388068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:13.388597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:13.390927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:13.431702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 812/1642] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:13.560949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:13.562492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:13.588094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 831/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:13.771599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8e10d-8ab8-44f3-a3af-df7039f1bb2c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:13.806844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8e10d-8ab8-44f3-a3af-df7039f1bb2c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:13.825270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:13.825823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:13.826101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:13.826388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:13.946391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:13.949277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:13.959345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:13.969031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 813/1644] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:14.180029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:14.190628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8506a59-1f6b-4d67-8144-8f13a3d9157f service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:14.202752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:14.203578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:14.203964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:14.204342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:14.219140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:14.223399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:14.247566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 832/1645] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:15.174757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:15.174757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171330a0-6405-47fd-9eb9-926793851766 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:15.174757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2eb0c-4843-431b-8465-e69d15452b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:15.174757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2eb0c-4843-431b-8465-e69d15452b63 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:15.186917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2eb0c-4843-431b-8465-e69d15452b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:15.186917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2eb0c-4843-431b-8465-e69d15452b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:15.191097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:15.191097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670fd8c-868a-437c-b40f-867d02995086 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:15.191097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654c06-1966-4cd2-b7d6-4558c49a23e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:15.191097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654c06-1966-4cd2-b7d6-4558c49a23e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:15.194459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654c06-1966-4cd2-b7d6-4558c49a23e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:15.194459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654c06-1966-4cd2-b7d6-4558c49a23e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:15.196824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:15.197102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654c06-1966-4cd2-b7d6-4558c49a23e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:15.197102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654c06-1966-4cd2-b7d6-4558c49a23e9 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:15.197102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 833/1646] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:15.198200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2eb0c-4843-431b-8465-e69d15452b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:15.198200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2eb0c-4843-431b-8465-e69d15452b63 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:15.198200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 814/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:15.419279 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb24283-b98a-4b6e-8401-4fd208ee0ba1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:15.419732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb24283-b98a-4b6e-8401-4fd208ee0ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:15.420336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb24283-b98a-4b6e-8401-4fd208ee0ba1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:15.422977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 834/1648] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 02:32:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:15.433976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19520b6-e625-411a-84ad-77b214c9c787 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:15.434545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19520b6-e625-411a-84ad-77b214c9c787 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:15.434835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19520b6-e625-411a-84ad-77b214c9c787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:15.470099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d19520b6-e625-411a-84ad-77b214c9c787 None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:15.471426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d19520b6-e625-411a-84ad-77b214c9c787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:15.482674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 815/1649] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 02:32:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:15.489728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ab9d-d59b-4a35-95d3-3537336885bc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:15.490682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ab9d-d59b-4a35-95d3-3537336885bc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:15.490969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ab9d-d59b-4a35-95d3-3537336885bc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:15.494131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 835/1650] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 02:32:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:15.507874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eeb8c-4557-4b19-ba38-d5b220241423 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:15.507874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eeb8c-4557-4b19-ba38-d5b220241423 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:15.507874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eeb8c-4557-4b19-ba38-d5b220241423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:15.538559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b04eeb8c-4557-4b19-ba38-d5b220241423 None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:15.540676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b04eeb8c-4557-4b19-ba38-d5b220241423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:15.565706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 816/1651] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:32:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:15.844286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3ab9d-d59b-4a35-95d3-3537336885bc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:15.885339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3ab9d-d59b-4a35-95d3-3537336885bc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:15.907299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:15.911696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:15.912105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:15.912384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:15.959650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:15.961101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:16.009140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 836/1652] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:18.429144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eeb8c-4557-4b19-ba38-d5b220241423 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:18.737134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eeb8c-4557-4b19-ba38-d5b220241423 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:18.777325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:18.777994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:18.778283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:18.778579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] PATH_INFO: `/v3/users/b532fee72b104c8094a63a738a37bf09` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:18.783059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b532fee72b104c8094a63a738a37bf09)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:18.797169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:19.082038 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:19.096309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a721636-477c-4ea0-b2b2-684d3c66b1dd service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:19.098318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b532fee72b104c8094a63a738a37bf09'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.102826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:19.104281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:19.104709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:19.105139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] PATH_INFO: `/v3/users/3a3b08b59b9c4e8ebe92821fe3de149e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:19.107288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3b08b59b9c4e8ebe92821fe3de149e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:19.108527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:32:19.108527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: warnings.warn(msg) Aug 08 02:32:19.108527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: Aug 08 02:32:19.111007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:19.182650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:19.215957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b532fee72b104c8094a63a738a37bf09'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.245216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b532fee72b104c8094a63a738a37bf09'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.277935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b532fee72b104c8094a63a738a37bf09'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.280270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ee3367cf-dbba-43d6-b5ae-fb48e51b5607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b532fee72b104c8094a63a738a37bf09'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.297240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 817/1653] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:18 2022] DELETE /identity/v3/users/b532fee72b104c8094a63a738a37bf09 => generated 0 bytes in 879 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:19.302214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:19.302618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:19.306094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:19.368088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:19.372133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:19.419891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:19.541694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3b08b59b9c4e8ebe92821fe3de149e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.563926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 818/1654] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:19.644591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3b08b59b9c4e8ebe92821fe3de149e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.672688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:19.674707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:19.708154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3b08b59b9c4e8ebe92821fe3de149e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.709545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3b08b59b9c4e8ebe92821fe3de149e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.712070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1c534484-dc4f-4adb-9dfb-c3caa0058209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3b08b59b9c4e8ebe92821fe3de149e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:19.781401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 837/1655] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:19 2022] DELETE /identity/v3/users/3a3b08b59b9c4e8ebe92821fe3de149e => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:19.784353 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:19.789699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:19.790530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:19.861839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23f307-117d-4b36-855e-f5cfd3ab5b31 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:19.869302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:19.877487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:20.021436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa386a-800e-4755-b777-b3fe70a7dbe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:20.021968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa386a-800e-4755-b777-b3fe70a7dbe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:20.022274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa386a-800e-4755-b777-b3fe70a7dbe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:20.022554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa386a-800e-4755-b777-b3fe70a7dbe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:20.025402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-20aa386a-800e-4755-b777-b3fe70a7dbe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:20.109491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 838/1656] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:20.127151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:20.130032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:20.152768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa386a-800e-4755-b777-b3fe70a7dbe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:20.154759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa386a-800e-4755-b777-b3fe70a7dbe6 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:20.160118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1ec0b-3dee-468d-8bf3-e4b735aa592a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:20.173923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 819/1657] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:20.175878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:20.176510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:20.176786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:20.177061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:20.194518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:20.196076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:20.231634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 839/1658] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:21.133308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:21.133767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:21.133854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:21.187255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] MFA Rules not processed for user `7fd9eb21aa634650a7022b7b4ca145fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:21.191038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:21.485133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 820/1659] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:21.519004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:21.520923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:21.654107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:21.655625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:21.712713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d86661-6580-415a-9d1e-3640cedf103d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:21.725159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:21.735553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d40e82-5b3f-4340-a351-bd1059a2b415 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:21.743987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:21.744974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:21.745389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:21.745838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:21.745955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:21.746702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:21.746929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:21.749373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] PATH_INFO: `/v3/projects/8dd587c816564259a98f16356ea478d0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:21.752064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd587c816564259a98f16356ea478d0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:21.757699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: WARNING py.warnings [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:32:21.757699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: warnings.warn(msg) Aug 08 02:32:21.757699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: Aug 08 02:32:21.759113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:21.761093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:21.768586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:21.782893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 840/1660] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:22.012700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd587c816564259a98f16356ea478d0', 'request_id': 'req-d4939908-7d71-4e24-8032-54f1478fb8f5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.056120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd587c816564259a98f16356ea478d0', 'request_id': 'req-d4939908-7d71-4e24-8032-54f1478fb8f5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.129407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd587c816564259a98f16356ea478d0', 'request_id': 'req-d4939908-7d71-4e24-8032-54f1478fb8f5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.152073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] The token cache is being invalidate because project 8dd587c816564259a98f16356ea478d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:22.153237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.154458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.158454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 821/1661] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:21 2022] DELETE /identity/v3/projects/8dd587c816564259a98f16356ea478d0 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:22.162957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:22.169402 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:22.179986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7b01d-9db8-4239-8b97-71fad8ab54fa service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:22.361458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:22.363018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:22.377555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:22.378219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:22.378528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:22.378809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] PATH_INFO: `/v3/users/194d0754979149fe937f8045097df030` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:22.386616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=194d0754979149fe937f8045097df030)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:22.393148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:22.395121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4939908-7d71-4e24-8032-54f1478fb8f5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:22.413597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:22.414992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:22.415138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:22.415326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] PATH_INFO: `/v3/projects/b86941379d2b4d7f81e0dd09a0494692` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:22.424910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b86941379d2b4d7f81e0dd09a0494692)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:22.431252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:22.677050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '194d0754979149fe937f8045097df030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.680839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86941379d2b4d7f81e0dd09a0494692', 'request_id': 'req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.690667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86941379d2b4d7f81e0dd09a0494692', 'request_id': 'req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.743598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '194d0754979149fe937f8045097df030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.766399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '194d0754979149fe937f8045097df030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.768311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '194d0754979149fe937f8045097df030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.768963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e414cac3-eb8d-4c0f-bf48-266cb8966af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '194d0754979149fe937f8045097df030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.790228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 841/1662] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:22 2022] DELETE /identity/v3/users/194d0754979149fe937f8045097df030 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:22.809213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86941379d2b4d7f81e0dd09a0494692', 'request_id': 'req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.838252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] The token cache is being invalidate because project b86941379d2b4d7f81e0dd09a0494692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:22.838637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.840043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb6c1a5b-be41-4dd9-80e0-28c6b4ef79d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:22.843367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 822/1663] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:22 2022] DELETE /identity/v3/projects/b86941379d2b4d7f81e0dd09a0494692 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:22.849970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17034c1-69f6-4677-9034-e228b209d5cb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:22.850404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17034c1-69f6-4677-9034-e228b209d5cb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:22.850818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17034c1-69f6-4677-9034-e228b209d5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:22.905971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e17034c1-69f6-4677-9034-e228b209d5cb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:22.907332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e17034c1-69f6-4677-9034-e228b209d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:23.077325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 842/1664] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:23.079705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14aea68-0590-40b6-8b41-d4e153cae092 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.080098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14aea68-0590-40b6-8b41-d4e153cae092 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.080522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14aea68-0590-40b6-8b41-d4e153cae092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.113464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c14aea68-0590-40b6-8b41-d4e153cae092 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:23.115046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c14aea68-0590-40b6-8b41-d4e153cae092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:23.142835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 823/1665] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:23.142917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17034c1-69f6-4677-9034-e228b209d5cb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:23.169839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17034c1-69f6-4677-9034-e228b209d5cb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:23.213564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:23.214169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.222248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.222248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.222248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:23.222248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:23.222248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:23.254187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 843/1666] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:23.259809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14aea68-0590-40b6-8b41-d4e153cae092 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:23.273050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14aea68-0590-40b6-8b41-d4e153cae092 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:23.286949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:23.288330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.288674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.289076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.290127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:23.292094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:23.338742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:23.479185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 824/1667] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:23 2022] POST /identity/v3/projects => generated 365 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:23.483445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:23.498675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456f27e-82e3-4f02-b147-d2c12ba50ce9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:23.717663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:23.719128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.719555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.719995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.720977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:23.726341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:23.777497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:23.872286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 844/1668] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:23 2022] POST /identity/v3/users => generated 283 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:23.873427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:23.894070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837587b-d437-4fce-be73-64c082306891 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:23.911294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:23.912159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.912614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.913027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.913825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:23.915897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:23.941588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 825/1669] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:23.946117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:23.965380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5767824-7d33-478a-b576-12e2a62489d1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:23.972344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:23.973365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.974614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.975660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.981385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:23.985591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:24.007495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:24.013840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 845/1670] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:24.013840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:24.014103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.039013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c642538-6544-4105-86be-8ac5984556b1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:24.046382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e1ca2-b2b2-4e7f-abb1-53b27f8d308d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:24.046597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d77197-81d6-4e87-80bf-ad620f8061eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:24.047288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d77197-81d6-4e87-80bf-ad620f8061eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.047611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d77197-81d6-4e87-80bf-ad620f8061eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.047891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d77197-81d6-4e87-80bf-ad620f8061eb admin admin] PATH_INFO: `/v3/projects/07fb905cb34d4975a028afc47b11f4d8/users/6cbbb2153e2c4448a53b89c407a14a6b/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.053962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:24.055391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.056137 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.056638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] PATH_INFO: `/v3/projects/4dbadb049eff4805bc99ed7515eb4088` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.059358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dbadb049eff4805bc99ed7515eb4088)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:24.062394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:24.079867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d77197-81d6-4e87-80bf-ad620f8061eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fb905cb34d4975a028afc47b11f4d8, user_id=6cbbb2153e2c4448a53b89c407a14a6b, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:24.082812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d77197-81d6-4e87-80bf-ad620f8061eb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:24.164981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 846/1671] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:24 2022] PUT /identity/v3/projects/07fb905cb34d4975a028afc47b11f4d8/users/6cbbb2153e2c4448a53b89c407a14a6b/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:24.178539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965532e-35e8-42c9-aa3e-6438aa352747 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.178807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965532e-35e8-42c9-aa3e-6438aa352747 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.179086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965532e-35e8-42c9-aa3e-6438aa352747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.235057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dbadb049eff4805bc99ed7515eb4088', 'request_id': 'req-f6620f8b-7887-4957-b8d0-e9e570df39b7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:24.236276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5965532e-35e8-42c9-aa3e-6438aa352747 None None] MFA Rules not processed for user `6cbbb2153e2c4448a53b89c407a14a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:24.237699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5965532e-35e8-42c9-aa3e-6438aa352747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.251619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dbadb049eff4805bc99ed7515eb4088', 'request_id': 'req-f6620f8b-7887-4957-b8d0-e9e570df39b7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:24.354611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dbadb049eff4805bc99ed7515eb4088', 'request_id': 'req-f6620f8b-7887-4957-b8d0-e9e570df39b7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:24.377450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] The token cache is being invalidate because project 4dbadb049eff4805bc99ed7515eb4088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:24.379153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:24.380228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f6620f8b-7887-4957-b8d0-e9e570df39b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:24.387109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 826/1672] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:24 2022] DELETE /identity/v3/projects/4dbadb049eff4805bc99ed7515eb4088 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:24.391312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.391312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.391312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.404548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 847/1673] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.406779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039c331-44e0-4287-99d3-860e22c800dc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.409191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039c331-44e0-4287-99d3-860e22c800dc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.409479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039c331-44e0-4287-99d3-860e22c800dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.467750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d039c331-44e0-4287-99d3-860e22c800dc None None] MFA Rules not processed for user `d725e4cb658b43438c1f22a80b1dc5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:24.469297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d039c331-44e0-4287-99d3-860e22c800dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.500220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] MFA Rules not processed for user `f6fb96d9442147e0a2201fe890d6f955`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:24.504167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.581494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 848/1674] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.584005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9ed9f-76d1-4bc4-bc29-0db4b425e095 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.584453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9ed9f-76d1-4bc4-bc29-0db4b425e095 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.584869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9ed9f-76d1-4bc4-bc29-0db4b425e095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.611105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f3b9ed9f-76d1-4bc4-bc29-0db4b425e095 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:24.614154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f3b9ed9f-76d1-4bc4-bc29-0db4b425e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.676098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 827/1675] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.677074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:24.684278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.699807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 849/1676] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.709431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffe14c-a500-4c05-bd2c-5c8162152dcd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.709836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffe14c-a500-4c05-bd2c-5c8162152dcd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.710215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffe14c-a500-4c05-bd2c-5c8162152dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.752641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ffffe14c-a500-4c05-bd2c-5c8162152dcd None None] MFA Rules not processed for user `d725e4cb658b43438c1f22a80b1dc5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:24.752641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ffffe14c-a500-4c05-bd2c-5c8162152dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.758007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c78db-3b1f-44d1-8465-f3b45a32f1dc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:24.768132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:24.770280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 850/1677] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.771821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e675e-6116-4a13-b691-329796850ab1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.772165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e675e-6116-4a13-b691-329796850ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.772446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e675e-6116-4a13-b691-329796850ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.776620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.776964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.777249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.782992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:24.785542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:24.809893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7e1e675e-6116-4a13-b691-329796850ab1 None None] MFA Rules not processed for user `f6fb96d9442147e0a2201fe890d6f955`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:24.811157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7e1e675e-6116-4a13-b691-329796850ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.822214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 851/1678] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.824418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.824776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.825128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.869569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:24.873076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.892733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 852/1679] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.896020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:24.901599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.950406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 828/1680] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/projects => generated 363 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:24.951584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:24.953864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:25.033322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed9434-0dce-4683-90f9-e0dbc06ba1fb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.045710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.102768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8a2f6-b774-419f-ba41-886283347bf8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.111434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.112692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.113371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.113831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.124828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.147146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.147699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.147987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.149009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.188375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.190580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.204040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 829/1681] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:25.205040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.225359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.227618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.252786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 853/1682] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:25.253748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.346101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd2d0-9dec-4010-a0f6-143f620b4555 service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.354868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.356735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.357180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.357648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.358470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:25.358907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.361387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.383899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 830/1683] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:25.388992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.413539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b841e34-daa3-4248-97b6-7116f5d794d8 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.430262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.430262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.430262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.430262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.430710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.436564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.440514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e2aed-17d5-4596-a383-79c4ca32c15a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.455561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.456179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.458604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.462248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.462971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.469252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.542074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 854/1684] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:25 2022] POST /identity/v3/users => generated 288 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:25.543149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.569492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659d90e-e519-426e-9409-3df6477586e1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.583651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.584398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.584798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.585136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.587686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.593103 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.597996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 831/1685] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:25 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:25.600543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.627604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 855/1686] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:25.631216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.661586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a213e-a93d-4120-9274-67e9848e9dd9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.725705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.734174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9f153-9dd5-4c15-8d44-e63e7f7ab011 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.741327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.742150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.743042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.743566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.744612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.747806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.833585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.839183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.839964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.840308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.841966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] PATH_INFO: `/v3/projects/93e9444a622d424fb310075577bea900/users/c5ef7275ff0247f8813b7a73ae1df726/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.873579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e9444a622d424fb310075577bea900, user_id=c5ef7275ff0247f8813b7a73ae1df726, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.876259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.906279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 832/1687] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:25 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:25.909273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.955244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579c5dd-fd45-4481-a322-e2c37453c18d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.962859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.963496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.963775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.964104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.965170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.967317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.997231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.997231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 856/1688] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:25 2022] PUT /identity/v3/projects/93e9444a622d424fb310075577bea900/users/c5ef7275ff0247f8813b7a73ae1df726/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:25.998158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.999045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 833/1689] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:26.001584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.036353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11570c38-1c92-4aca-920e-2895e379e5ba admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.154112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede61dd-cf8d-429e-89a7-69b6c24d6829 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.161948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dc15f-478f-43c7-a783-99f2cd32a8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.165006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dc15f-478f-43c7-a783-99f2cd32a8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.166450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dc15f-478f-43c7-a783-99f2cd32a8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.167050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dc15f-478f-43c7-a783-99f2cd32a8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.168014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dc15f-478f-43c7-a783-99f2cd32a8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.170193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dc15f-478f-43c7-a783-99f2cd32a8c3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.181504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf39b2c-67c7-4b22-bdb0-d5a12af7de3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.182172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf39b2c-67c7-4b22-bdb0-d5a12af7de3f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.182824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf39b2c-67c7-4b22-bdb0-d5a12af7de3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.183425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf39b2c-67c7-4b22-bdb0-d5a12af7de3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.184234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf39b2c-67c7-4b22-bdb0-d5a12af7de3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.188814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf39b2c-67c7-4b22-bdb0-d5a12af7de3f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.189627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 857/1690] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:26.192009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e796ce-cb86-4a20-9f1e-06e0d1dbfb7b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.192520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e796ce-cb86-4a20-9f1e-06e0d1dbfb7b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.192997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e796ce-cb86-4a20-9f1e-06e0d1dbfb7b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.195727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 858/1691] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 02:32:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:26.196800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e796ce-cb86-4a20-9f1e-06e0d1dbfb7b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.207787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 834/1692] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:26.209743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740fb57-c484-4e41-a1d1-6688f3e05c9e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.210112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740fb57-c484-4e41-a1d1-6688f3e05c9e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.211083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740fb57-c484-4e41-a1d1-6688f3e05c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.213643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e796ce-cb86-4a20-9f1e-06e0d1dbfb7b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.219354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09206531-21f1-4f98-ab4e-e6a1b6b4a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.223009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09206531-21f1-4f98-ab4e-e6a1b6b4a6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.223517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09206531-21f1-4f98-ab4e-e6a1b6b4a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.223970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09206531-21f1-4f98-ab4e-e6a1b6b4a6d6 admin admin] PATH_INFO: `/v3/projects/cd638bc9d37443299c13a1f6387aebce/users/573ba5c44804441d86e5c04bbbf67f0b/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.249482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09206531-21f1-4f98-ab4e-e6a1b6b4a6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd638bc9d37443299c13a1f6387aebce, user_id=573ba5c44804441d86e5c04bbbf67f0b, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.254176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09206531-21f1-4f98-ab4e-e6a1b6b4a6d6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.270618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f740fb57-c484-4e41-a1d1-6688f3e05c9e None None] MFA Rules not processed for user `e8839af749e44fac8a7a311b61463c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:26.270618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f740fb57-c484-4e41-a1d1-6688f3e05c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.360578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 835/1693] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:26.361930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740fb57-c484-4e41-a1d1-6688f3e05c9e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.371071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 859/1694] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:26 2022] PUT /identity/v3/projects/cd638bc9d37443299c13a1f6387aebce/users/573ba5c44804441d86e5c04bbbf67f0b/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:26.382991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9a1c-260c-4f3c-bd0d-7eb89d10db9a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.383447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9a1c-260c-4f3c-bd0d-7eb89d10db9a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.383783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c9a1c-260c-4f3c-bd0d-7eb89d10db9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.415535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740fb57-c484-4e41-a1d1-6688f3e05c9e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.452326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-205c9a1c-260c-4f3c-bd0d-7eb89d10db9a None None] MFA Rules not processed for user `573ba5c44804441d86e5c04bbbf67f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:26.453499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-205c9a1c-260c-4f3c-bd0d-7eb89d10db9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.488993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.489883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.490300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.490700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] PATH_INFO: `/v3/projects/93e9444a622d424fb310075577bea900/users/c5ef7275ff0247f8813b7a73ae1df726/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.494458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e9444a622d424fb310075577bea900, user_id=c5ef7275ff0247f8813b7a73ae1df726, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.498020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.499695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:26.533412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 836/1695] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:26 2022] PUT /identity/v3/projects/93e9444a622d424fb310075577bea900/users/c5ef7275ff0247f8813b7a73ae1df726/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:26.534513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.536354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.614407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 860/1696] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:26.615775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15af3cf-4a21-48f9-a5e9-3b88965f94ea None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.616099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15af3cf-4a21-48f9-a5e9-3b88965f94ea None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.616330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15af3cf-4a21-48f9-a5e9-3b88965f94ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.638405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c15af3cf-4a21-48f9-a5e9-3b88965f94ea None None] MFA Rules not processed for user `7fd9eb21aa634650a7022b7b4ca145fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:26.639398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef224f7a-3ee4-4cf6-a6df-b36e29acf952 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.639629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c15af3cf-4a21-48f9-a5e9-3b88965f94ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.644688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc25505-6ed1-4c43-bf3b-0f7a617f3cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.650073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc25505-6ed1-4c43-bf3b-0f7a617f3cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.650073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc25505-6ed1-4c43-bf3b-0f7a617f3cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.650073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc25505-6ed1-4c43-bf3b-0f7a617f3cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.655451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc25505-6ed1-4c43-bf3b-0f7a617f3cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.657121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc25505-6ed1-4c43-bf3b-0f7a617f3cc9 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.667081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 837/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:26.668652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfa982-bf1d-4b79-8aa5-fd573c44ab9e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.669062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfa982-bf1d-4b79-8aa5-fd573c44ab9e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.669414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfa982-bf1d-4b79-8aa5-fd573c44ab9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.728880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 861/1698] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:26.730918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74b551-3445-4647-b99f-cf6d316b6478 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.731400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74b551-3445-4647-b99f-cf6d316b6478 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.731880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74b551-3445-4647-b99f-cf6d316b6478 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.735214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 862/1699] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 02:32:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:26.736553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74b551-3445-4647-b99f-cf6d316b6478 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.739505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e74b551-3445-4647-b99f-cf6d316b6478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.759804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a0cfa982-bf1d-4b79-8aa5-fd573c44ab9e None None] MFA Rules not processed for user `c5ef7275ff0247f8813b7a73ae1df726`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:26.761165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a0cfa982-bf1d-4b79-8aa5-fd573c44ab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.770668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:26.831634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74b551-3445-4647-b99f-cf6d316b6478 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.841235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.842707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.843168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.843621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.909353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 838/1700] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:26.910799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.911086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.911361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.949454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] MFA Rules not processed for user `e8839af749e44fac8a7a311b61463c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:26.951354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.960264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 839/1701] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:26.962474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.963688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.964271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.965137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.987609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877811e-f8e1-49b3-beca-dda1789c07a0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.988596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 863/1702] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:26.989350 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.991718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.992771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.993734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.994089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.996629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.016995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943b217-d1d9-4738-a6eb-153863322327 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.022727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.023841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.024415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.024898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.047316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.049654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.059608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.061079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.065351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 864/1703] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:27.070844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.113804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 840/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:27.168901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014b1b2-3fa0-4f5b-a7c4-21cdcffe45ca service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.175934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.184042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.184042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.184042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] PATH_INFO: `/v3/users/573ba5c44804441d86e5c04bbbf67f0b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.184042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] RBAC: Authorizing `identity:delete_user(user_id=573ba5c44804441d86e5c04bbbf67f0b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.191324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.381429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.394178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10598773-0fac-46c1-832b-029cdce34f66 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.403300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56293c1d-d6ff-4ec2-a695-b2fad973490e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.404101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56293c1d-d6ff-4ec2-a695-b2fad973490e service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.404471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56293c1d-d6ff-4ec2-a695-b2fad973490e service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.404811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56293c1d-d6ff-4ec2-a695-b2fad973490e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.411789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-56293c1d-d6ff-4ec2-a695-b2fad973490e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:27.462061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573ba5c44804441d86e5c04bbbf67f0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:27.467898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:27.558524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573ba5c44804441d86e5c04bbbf67f0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:27.579419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:27.580875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573ba5c44804441d86e5c04bbbf67f0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:27.581907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573ba5c44804441d86e5c04bbbf67f0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:27.582458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-249688dd-e321-4ac6-bef4-315ea431d37c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573ba5c44804441d86e5c04bbbf67f0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:27.588764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56293c1d-d6ff-4ec2-a695-b2fad973490e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.590643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56293c1d-d6ff-4ec2-a695-b2fad973490e service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.615291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 865/1705] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:27 2022] DELETE /identity/v3/users/573ba5c44804441d86e5c04bbbf67f0b => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:27.619644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190693c-b815-427f-898e-a26e578fa26b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.619949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190693c-b815-427f-898e-a26e578fa26b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.620226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190693c-b815-427f-898e-a26e578fa26b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.620778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 841/1706] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:27.639597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 866/1707] 10.222.0.15 () {54 vars in 961 bytes} [Mon Aug 8 02:32:27 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:27.639793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.640396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.653735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.655601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0636-6497-4bca-ac36-f7db69a27032 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.655885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0636-6497-4bca-ac36-f7db69a27032 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.656162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0636-6497-4bca-ac36-f7db69a27032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.733429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:27.735764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:27.745050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b3bf0636-6497-4bca-ac36-f7db69a27032 None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:27.746294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b3bf0636-6497-4bca-ac36-f7db69a27032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:27.877247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 867/1708] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Aug 8 02:32:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:27.886584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0636-6497-4bca-ac36-f7db69a27032 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.911934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0636-6497-4bca-ac36-f7db69a27032 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.938885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ca2f7-ddb8-4281-8c34-175b6c22652c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.940680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca2f7-ddb8-4281-8c34-175b6c22652c service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.941153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca2f7-ddb8-4281-8c34-175b6c22652c service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.941778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca2f7-ddb8-4281-8c34-175b6c22652c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.944154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c2ca2f7-ddb8-4281-8c34-175b6c22652c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:27.955682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 842/1709] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:27.969013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ca2f7-ddb8-4281-8c34-175b6c22652c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.969336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.971372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ca2f7-ddb8-4281-8c34-175b6c22652c service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.972509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:28.009994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 868/1710] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:28.216435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608e032-fd16-4ae3-876a-01d2d9a46814 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:28.231615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:28.249301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.249301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.249301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.274664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:28.276277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.288549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 843/1711] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:28.420660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d8335-f3d4-42d6-9814-73f41e59c510 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.420964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d8335-f3d4-42d6-9814-73f41e59c510 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.421243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d8335-f3d4-42d6-9814-73f41e59c510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.461295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1b7d8335-f3d4-42d6-9814-73f41e59c510 None None] MFA Rules not processed for user `f6fb96d9442147e0a2201fe890d6f955`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:28.462597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1b7d8335-f3d4-42d6-9814-73f41e59c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:28.605036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 869/1712] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:29.281698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:29.285636 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:29.320240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49306f9-abc6-42b8-a8c4-a4bc1b917e58 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:29.339263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:29.345354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:29.349166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:29.349478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] PATH_INFO: `/v3/projects/cd638bc9d37443299c13a1f6387aebce` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:29.353198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd638bc9d37443299c13a1f6387aebce)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:29.358692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:29.510539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd638bc9d37443299c13a1f6387aebce', 'request_id': 'req-d13a5f1b-a140-4767-b70f-c122584684c2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:29.518529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd638bc9d37443299c13a1f6387aebce', 'request_id': 'req-d13a5f1b-a140-4767-b70f-c122584684c2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:29.604270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd638bc9d37443299c13a1f6387aebce', 'request_id': 'req-d13a5f1b-a140-4767-b70f-c122584684c2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:29.638511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] The token cache is being invalidate because project cd638bc9d37443299c13a1f6387aebce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:29.638820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:29.643335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d13a5f1b-a140-4767-b70f-c122584684c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:29.648756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 844/1713] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:29 2022] DELETE /identity/v3/projects/cd638bc9d37443299c13a1f6387aebce => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:29.655877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357eb6d-4af6-4f52-8866-eb025121a725 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:29.657362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357eb6d-4af6-4f52-8866-eb025121a725 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:29.657884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357eb6d-4af6-4f52-8866-eb025121a725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:29.704881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7357eb6d-4af6-4f52-8866-eb025121a725 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:29.706462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7357eb6d-4af6-4f52-8866-eb025121a725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:29.863690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:29.888800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40687f9c-99c7-44a0-a8da-6a146239c401 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:29.889120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40687f9c-99c7-44a0-a8da-6a146239c401 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:29.889360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40687f9c-99c7-44a0-a8da-6a146239c401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:29.890492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 870/1714] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:29.961334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-40687f9c-99c7-44a0-a8da-6a146239c401 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:29.962919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-40687f9c-99c7-44a0-a8da-6a146239c401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:30.000242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 845/1715] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:30.003903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357eb6d-4af6-4f52-8866-eb025121a725 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:30.018004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357eb6d-4af6-4f52-8866-eb025121a725 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:30.028727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:30.029743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:30.030226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:30.030731 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:30.036743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:30.037728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:30.046487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:30.083850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 871/1716] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:30.088583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40687f9c-99c7-44a0-a8da-6a146239c401 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:30.110400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40687f9c-99c7-44a0-a8da-6a146239c401 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:30.131471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:30.132654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:30.133108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:30.133888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:30.134753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:30.141076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:30.229736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:30.352640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 846/1717] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:30 2022] POST /identity/v3/projects => generated 375 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:30.356835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:30.373094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e9090-8bdc-41e4-a3a4-0591a55fed93 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:30.525378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:30.527720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:30.528282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:30.529271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:30.530908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:30.535769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:30.690704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 872/1718] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:30 2022] POST /identity/v3/users => generated 288 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:30.694736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:30.722623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c224ad-ddea-408c-bbd8-e6f85ce33450 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:30.731424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:30.732071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:30.732519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:30.732968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:30.733757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:30.736055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:30.769207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 847/1719] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:30.773288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:30.788255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829d63-f9e2-478c-9f06-81985d0aceca admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:30.803085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437eadc9-f46f-4741-af7b-e17e512218de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:30.805012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eadc9-f46f-4741-af7b-e17e512218de admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:30.805672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eadc9-f46f-4741-af7b-e17e512218de admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:30.806132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437eadc9-f46f-4741-af7b-e17e512218de admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:30.807012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437eadc9-f46f-4741-af7b-e17e512218de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:30.808808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437eadc9-f46f-4741-af7b-e17e512218de admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:30.833857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 873/1720] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:30.838427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:30.852995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176d072-86e9-455b-902d-cea88ed29ca5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:30.906237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:30.907178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:30.907677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:30.909227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] PATH_INFO: `/v3/projects/7f65d1049f874d27b9e3ef2bdebd1535/users/ade87b4356014d2497971dc70380ed52/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:30.972264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f65d1049f874d27b9e3ef2bdebd1535, user_id=ade87b4356014d2497971dc70380ed52, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:30.973403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:31.125247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 848/1721] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:30 2022] PUT /identity/v3/projects/7f65d1049f874d27b9e3ef2bdebd1535/users/ade87b4356014d2497971dc70380ed52/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:31.138668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:31.141144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:31.141687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:31.227610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] MFA Rules not processed for user `ade87b4356014d2497971dc70380ed52`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:31.231684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:31.404496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 874/1722] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:31.416614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:31.419283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:31.548408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6337f-8d2c-483e-a341-f0e076ddfa9f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:31.556397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fbfe8-69e9-4eac-affa-056abec095ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:31.557073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fbfe8-69e9-4eac-affa-056abec095ec service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:31.557461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fbfe8-69e9-4eac-affa-056abec095ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:31.557742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fbfe8-69e9-4eac-affa-056abec095ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:31.571942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fbfe8-69e9-4eac-affa-056abec095ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:31.573860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fbfe8-69e9-4eac-affa-056abec095ec service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:31.589263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 849/1723] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:31.680952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:31.683486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:31.787210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:31.825468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90b168-27de-4a5b-a3e3-572c253a08d4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:31.833691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:31.835082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:31.842530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:31.842530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] PATH_INFO: `/v3/users/004ac2aedb8648139be9c3f04653aa16` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:31.842530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=004ac2aedb8648139be9c3f04653aa16)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:31.843865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.117888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004ac2aedb8648139be9c3f04653aa16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:32.213824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004ac2aedb8648139be9c3f04653aa16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:32.230089 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004ac2aedb8648139be9c3f04653aa16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:32.230902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004ac2aedb8648139be9c3f04653aa16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:32.231533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004ac2aedb8648139be9c3f04653aa16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:32.247929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 875/1724] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:31 2022] DELETE /identity/v3/users/004ac2aedb8648139be9c3f04653aa16 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:32.252314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.252602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.252880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.271690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:32.273072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:32.334235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 850/1725] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:32.344053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.345623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:32.404592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddb5d7-365a-4bb3-8089-07a4684c7ef8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.408225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.408825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.409102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.409373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.420911 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.422343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.442080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 876/1726] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:33.018442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:33.020888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.043590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e82157-391c-4ba3-bb33-15b6d6388fbd None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:33.058260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:33.061706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.062218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.062720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] PATH_INFO: `/v3/users/6cbbb2153e2c4448a53b89c407a14a6b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.064581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbbb2153e2c4448a53b89c407a14a6b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:33.067234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:33.380166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbbb2153e2c4448a53b89c407a14a6b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.434703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbbb2153e2c4448a53b89c407a14a6b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.434968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:33.437106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.437610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:33.455295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbbb2153e2c4448a53b89c407a14a6b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.457795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac0fa-056f-427d-bab4-8a612a5311c8 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:33.464832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:33.465371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbbb2153e2c4448a53b89c407a14a6b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.466054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d505da6f-0cf7-4f06-ade2-495060d40af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbbb2153e2c4448a53b89c407a14a6b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.466240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.466757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.467237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] PATH_INFO: `/v3/projects/84bd2e47b642474f8f51f028747a8e77` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.468552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84bd2e47b642474f8f51f028747a8e77)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:33.471343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:33.472064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 851/1727] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:33 2022] DELETE /identity/v3/users/6cbbb2153e2c4448a53b89c407a14a6b => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:33.480052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.481761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.481761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.514450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:33.515798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.617612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 852/1728] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:33.625241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:33.626823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.681824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84bd2e47b642474f8f51f028747a8e77', 'request_id': 'req-b5c6f27d-7161-4648-beae-6646c9c95696'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.756534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce2707-08cc-4750-87f0-a5c23b860c33 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:33.784909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84bd2e47b642474f8f51f028747a8e77', 'request_id': 'req-b5c6f27d-7161-4648-beae-6646c9c95696'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.815479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84bd2e47b642474f8f51f028747a8e77', 'request_id': 'req-b5c6f27d-7161-4648-beae-6646c9c95696'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.837367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:33.855456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] The token cache is being invalidate because project 84bd2e47b642474f8f51f028747a8e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:33.855751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.856523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5c6f27d-7161-4648-beae-6646c9c95696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.864433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 877/1729] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:33 2022] DELETE /identity/v3/projects/84bd2e47b642474f8f51f028747a8e77 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:33.871510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851067a-b9cb-46bd-97be-1e222bdffc40 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.871799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851067a-b9cb-46bd-97be-1e222bdffc40 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.872077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851067a-b9cb-46bd-97be-1e222bdffc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.880721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78acc24-ceec-494c-be10-51c4ab414a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:33.881972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78acc24-ceec-494c-be10-51c4ab414a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.882544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78acc24-ceec-494c-be10-51c4ab414a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.884314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78acc24-ceec-494c-be10-51c4ab414a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.886715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c78acc24-ceec-494c-be10-51c4ab414a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.928528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0851067a-b9cb-46bd-97be-1e222bdffc40 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:33.953711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0851067a-b9cb-46bd-97be-1e222bdffc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:34.035672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:34.091406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78acc24-ceec-494c-be10-51c4ab414a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:34.092904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78acc24-ceec-494c-be10-51c4ab414a85 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.127991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 853/1730] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:34.129485 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a11aa8-bf6a-4e90-8756-c40ad30d8c6f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.129727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a11aa8-bf6a-4e90-8756-c40ad30d8c6f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.130006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a11aa8-bf6a-4e90-8756-c40ad30d8c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.150520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 878/1731] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:34.158040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd761d-4b5d-4aa6-bf37-ce41e1d5aa0f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.158393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd761d-4b5d-4aa6-bf37-ce41e1d5aa0f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.159065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd761d-4b5d-4aa6-bf37-ce41e1d5aa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.186125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-28dd761d-4b5d-4aa6-bf37-ce41e1d5aa0f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:34.187434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-28dd761d-4b5d-4aa6-bf37-ce41e1d5aa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:34.203112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 879/1732] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:34.205729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd761d-4b5d-4aa6-bf37-ce41e1d5aa0f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.232884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-58a11aa8-bf6a-4e90-8756-c40ad30d8c6f None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:34.238597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-58a11aa8-bf6a-4e90-8756-c40ad30d8c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:34.266939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd761d-4b5d-4aa6-bf37-ce41e1d5aa0f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:34.285954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:34.286736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.287339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.287339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.289122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:34.289584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:34.291327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.323339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 880/1733] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:34.327462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.344541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a074540-a3d6-4298-b5e3-fb824ac0edbc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:34.360603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:34.361217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.361495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.361940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.362599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:34.365628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.448904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 854/1734] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:32:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:34.509361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 881/1735] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:34 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:34.513639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a11aa8-bf6a-4e90-8756-c40ad30d8c6f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.549340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a11aa8-bf6a-4e90-8756-c40ad30d8c6f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:34.708484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:34.708484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.708484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.708484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.708894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:34.708894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.803111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 855/1736] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:34 2022] POST /identity/v3/users => generated 288 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:34.804683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.842698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e11bb5-531f-4ba0-96a1-a31790e40360 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:34.865134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:34.865908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.866606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.867105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.868196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:34.870073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.896966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 882/1737] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:34.898974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.919577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef86ff-8fe5-439a-86e2-77c8026cd90f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:34.934594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:34.935318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.935691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.936061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.936965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:34.939821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.965237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.966509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 856/1738] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:34.968833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:34.971918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.994199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190e14e-7403-4723-b428-57ae7780254a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.008174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0c165-14d2-472a-886e-7ccaa32e6e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.009178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0c165-14d2-472a-886e-7ccaa32e6e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.009551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0c165-14d2-472a-886e-7ccaa32e6e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.009921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0c165-14d2-472a-886e-7ccaa32e6e03 admin admin] PATH_INFO: `/v3/projects/94cdabae2f2f4114b107d1ef4c49ba7a/users/06717ab36e1c4265beab5e6213f37eb3/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.051862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0c165-14d2-472a-886e-7ccaa32e6e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cdabae2f2f4114b107d1ef4c49ba7a, user_id=06717ab36e1c4265beab5e6213f37eb3, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.055989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0c165-14d2-472a-886e-7ccaa32e6e03 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.176135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 857/1739] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:34 2022] PUT /identity/v3/projects/94cdabae2f2f4114b107d1ef4c49ba7a/users/06717ab36e1c4265beab5e6213f37eb3/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:35.196587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.197023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.197298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.214949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:35.224550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc591651-f56b-4a2e-a094-f0c482050808 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.243775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] MFA Rules not processed for user `06717ab36e1c4265beab5e6213f37eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:35.245324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:35.256515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:35.331928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.332380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.332663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.332947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.356796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 858/1740] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:35.374063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:35.379322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:35.455739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.457795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.483770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 883/1741] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:32:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:35.524495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10be57-24f0-446e-9145-6f3533df01e0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.537786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aabb2-7f13-4a0d-b4e1-52abbeb018d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.539217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aabb2-7f13-4a0d-b4e1-52abbeb018d2 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.540575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aabb2-7f13-4a0d-b4e1-52abbeb018d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.540575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aabb2-7f13-4a0d-b4e1-52abbeb018d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.552500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aabb2-7f13-4a0d-b4e1-52abbeb018d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.554148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aabb2-7f13-4a0d-b4e1-52abbeb018d2 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.612744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 859/1742] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:35.619193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:35.622659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:35.887939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336f4c7-d3ab-4bcd-b7a3-aca4d2ff97fa service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.903114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.903114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.910313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.910313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] PATH_INFO: `/v3/projects/07fb905cb34d4975a028afc47b11f4d8` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.910313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fb905cb34d4975a028afc47b11f4d8)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.916815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.187583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fb905cb34d4975a028afc47b11f4d8', 'request_id': 'req-1ae5facc-365d-4b47-9690-364719e5b2d2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.296069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.297070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.297435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.321380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fb905cb34d4975a028afc47b11f4d8', 'request_id': 'req-1ae5facc-365d-4b47-9690-364719e5b2d2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.348643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fb905cb34d4975a028afc47b11f4d8', 'request_id': 'req-1ae5facc-365d-4b47-9690-364719e5b2d2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.353077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] MFA Rules not processed for user `e8839af749e44fac8a7a311b61463c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:36.356900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.362699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:36.378504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] The token cache is being invalidate because project 07fb905cb34d4975a028afc47b11f4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:36.378849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.392345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.404755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 884/1743] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:35 2022] DELETE /identity/v3/projects/07fb905cb34d4975a028afc47b11f4d8 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:36.405897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.410206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.487184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 860/1744] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 8 02:32:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:36.504752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.506759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.522666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c87b6-8efc-4557-8743-e2921ec73aec None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.527825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.529091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.538038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.539717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.552735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.555405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.568955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 861/1745] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:36.582089 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5facc-365d-4b47-9690-364719e5b2d2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.595900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.596616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.596852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.597172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] PATH_INFO: `/v3/users/c5ef7275ff0247f8813b7a73ae1df726` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.599319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ef7275ff0247f8813b7a73ae1df726)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.606312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.637449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.638938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.686593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd61fc-64d5-473c-8ac5-6fd214c2775b service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.721946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.724456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.726860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.728622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] PATH_INFO: `/v3/users/06717ab36e1c4265beab5e6213f37eb3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.737310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06717ab36e1c4265beab5e6213f37eb3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.739574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.791263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:36.848436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ef7275ff0247f8813b7a73ae1df726'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.941251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ef7275ff0247f8813b7a73ae1df726'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.979137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ef7275ff0247f8813b7a73ae1df726'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.980533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ef7275ff0247f8813b7a73ae1df726'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.981611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ef7275ff0247f8813b7a73ae1df726'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.025961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 885/1746] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:36 2022] DELETE /identity/v3/users/c5ef7275ff0247f8813b7a73ae1df726 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:37.060740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06717ab36e1c4265beab5e6213f37eb3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.809979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06717ab36e1c4265beab5e6213f37eb3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.843358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06717ab36e1c4265beab5e6213f37eb3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.862938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:37.864943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:37.870832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06717ab36e1c4265beab5e6213f37eb3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.870982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e804a7f-2d20-44bf-aa3f-4b876908d425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06717ab36e1c4265beab5e6213f37eb3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.898116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 862/1747] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:36 2022] DELETE /identity/v3/users/06717ab36e1c4265beab5e6213f37eb3 => generated 0 bytes in 1259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:37.908937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.910253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.910603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.982767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:37.984114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:38.070153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccc07d-a473-405d-87a0-5e2b71968b6d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.169793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:38.175818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 863/1748] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:38.268784 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.270747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:38.343117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.344312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.352630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.352988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] PATH_INFO: `/v3/users/ade87b4356014d2497971dc70380ed52` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.360485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade87b4356014d2497971dc70380ed52)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.364465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.484362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30072849-66cf-4397-a0b7-11819ff92590 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.491251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c80da-e948-4999-9b96-a116681b68e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.494701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c80da-e948-4999-9b96-a116681b68e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.494701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c80da-e948-4999-9b96-a116681b68e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.494701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c80da-e948-4999-9b96-a116681b68e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.518643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c80da-e948-4999-9b96-a116681b68e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.523296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c80da-e948-4999-9b96-a116681b68e6 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.550020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 864/1749] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:38.681839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade87b4356014d2497971dc70380ed52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:38.757843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade87b4356014d2497971dc70380ed52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:38.810312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade87b4356014d2497971dc70380ed52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:38.811817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade87b4356014d2497971dc70380ed52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:38.811817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-21aa64af-3166-4bfb-b254-5fa19e91a841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade87b4356014d2497971dc70380ed52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:38.846407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 886/1750] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:37 2022] DELETE /identity/v3/users/ade87b4356014d2497971dc70380ed52 => generated 0 bytes in 984 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:38.851503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.852514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.852855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.874152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca354cd5-3bcd-4061-bdac-a6feaef257b7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.874754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca354cd5-3bcd-4061-bdac-a6feaef257b7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.874852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca354cd5-3bcd-4061-bdac-a6feaef257b7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.880672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 887/1751] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:32:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:38.900465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.900842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.901158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.918294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:38.919679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:38.957012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:38.982735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:38.985443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.022731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 865/1752] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:39.093835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 888/1753] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:32:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:39.150149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.151636 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.317650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388492-60db-4a34-ad6f-542d83a3d563 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.332476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.336610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.336610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.336610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.374701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.376173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.403196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 866/1754] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:39.418330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.420024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.437691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.442390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.470884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa6033-d15f-4b9b-b83c-197a87f9bbc3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.480175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb7711-c3d9-4ed6-9787-a28caa289029 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.487651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9987-812e-41cc-b5c6-8c45b1aa1d44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.488215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.488739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9987-812e-41cc-b5c6-8c45b1aa1d44 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.489261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9987-812e-41cc-b5c6-8c45b1aa1d44 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.489754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9987-812e-41cc-b5c6-8c45b1aa1d44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.506145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.506524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.509550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] PATH_INFO: `/v3/projects/93e9444a622d424fb310075577bea900` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.511231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9987-812e-41cc-b5c6-8c45b1aa1d44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.513051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9987-812e-41cc-b5c6-8c45b1aa1d44 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.515641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e9444a622d424fb310075577bea900)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.521018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.549010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 889/1755] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:39.710780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e9444a622d424fb310075577bea900', 'request_id': 'req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.726426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e9444a622d424fb310075577bea900', 'request_id': 'req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.845057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e9444a622d424fb310075577bea900', 'request_id': 'req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.890752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] The token cache is being invalidate because project 93e9444a622d424fb310075577bea900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:39.891389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.891771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3b93fbd0-9d46-42b4-98a9-a0dcaae373b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.904308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 867/1756] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:39 2022] DELETE /identity/v3/projects/93e9444a622d424fb310075577bea900 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:39.904489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34f4b4-3ac6-49c7-b8ea-2797f1fa508b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.904489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34f4b4-3ac6-49c7-b8ea-2797f1fa508b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.904489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34f4b4-3ac6-49c7-b8ea-2797f1fa508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.953151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bf34f4b4-3ac6-49c7-b8ea-2797f1fa508b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:39.966941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bf34f4b4-3ac6-49c7-b8ea-2797f1fa508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:40.026032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:40.127831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 890/1757] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:40.137266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8bd25-3f8f-4279-bb6e-5298586881fb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.137666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8bd25-3f8f-4279-bb6e-5298586881fb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.138004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8bd25-3f8f-4279-bb6e-5298586881fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.165508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-95a8bd25-3f8f-4279-bb6e-5298586881fb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:40.169555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-95a8bd25-3f8f-4279-bb6e-5298586881fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:40.175753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 868/1758] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:40.181057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34f4b4-3ac6-49c7-b8ea-2797f1fa508b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.193450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34f4b4-3ac6-49c7-b8ea-2797f1fa508b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.199879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:40.200853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.201199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.201564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.202603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:40.203010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:40.204917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:40.225207 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8bd25-3f8f-4279-bb6e-5298586881fb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.226661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 891/1759] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:40.238592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8bd25-3f8f-4279-bb6e-5298586881fb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.256560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:40.258004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.258743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.259204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.260386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:40.263377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:40.419265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.423762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:40.433911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 869/1760] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/projects => generated 401 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:40.439414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.470464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178a94c-4176-4b2a-ba04-ac175c3f36eb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.553499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:40.629682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:40.630396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.646437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.646437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] PATH_INFO: `/v3/projects/94cdabae2f2f4114b107d1ef4c49ba7a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.650453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cdabae2f2f4114b107d1ef4c49ba7a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:40.652560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:40.657058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22542c9-f6fb-40ef-9a1c-2c81477f4d47 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.687168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:40.687809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.688911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.689899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.691135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:40.696543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:40.818560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cdabae2f2f4114b107d1ef4c49ba7a', 'request_id': 'req-1158b54a-e83a-4886-af1c-aadae2a6394e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.880593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 870/1761] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/users => generated 301 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:40.883366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.895617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cdabae2f2f4114b107d1ef4c49ba7a', 'request_id': 'req-1158b54a-e83a-4886-af1c-aadae2a6394e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.913902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cdabae2f2f4114b107d1ef4c49ba7a', 'request_id': 'req-1158b54a-e83a-4886-af1c-aadae2a6394e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.926527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49914bd-512d-41b7-994a-ef4d07290ee7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.956363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] The token cache is being invalidate because project 94cdabae2f2f4114b107d1ef4c49ba7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:40.956934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.966189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1158b54a-e83a-4886-af1c-aadae2a6394e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.970286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 892/1762] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:40 2022] DELETE /identity/v3/projects/94cdabae2f2f4114b107d1ef4c49ba7a => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:40.975330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811601d-e4ef-4eef-b4f6-5b1e0c0c3819 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.976117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811601d-e4ef-4eef-b4f6-5b1e0c0c3819 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.979991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811601d-e4ef-4eef-b4f6-5b1e0c0c3819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.077833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5811601d-e4ef-4eef-b4f6-5b1e0c0c3819 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:41.083738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5811601d-e4ef-4eef-b4f6-5b1e0c0c3819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.198106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.199793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.200068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.200356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.200999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.214726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.223453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:41.250056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 871/1763] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:41.255366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.263295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.307083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a3baf-0ab8-43c0-9d2f-e481ea6f3c51 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.319049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.321272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.321721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.322712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.323306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 893/1764] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:41.324858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfa336-dd17-4ec4-a0ec-9abde555f06b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.325253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfa336-dd17-4ec4-a0ec-9abde555f06b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.325622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfa336-dd17-4ec4-a0ec-9abde555f06b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.326107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.330587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.336921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 894/1765] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:32:41 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:41.338591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f6c3-b52a-49e2-8be0-4d4ece43ba30 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.338971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f6c3-b52a-49e2-8be0-4d4ece43ba30 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.339579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f6c3-b52a-49e2-8be0-4d4ece43ba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.367453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 872/1766] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:41.369435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.373319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.377565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6ca7f6c3-b52a-49e2-8be0-4d4ece43ba30 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:41.379633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6ca7f6c3-b52a-49e2-8be0-4d4ece43ba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.399743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 895/1767] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:41.401398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f6c3-b52a-49e2-8be0-4d4ece43ba30 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.414283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a276665-19a1-4cd7-aef0-530c55938631 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.416248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f6c3-b52a-49e2-8be0-4d4ece43ba30 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.424689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.425598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.426336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.426775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] PATH_INFO: `/v3/projects/a0f00624cad6436f9629589de0ca1962/users/37be0b331eec44b3a5827d99e72b3860/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.436384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.437032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.437305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.437670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] PATH_INFO: `/v3/projects/7f65d1049f874d27b9e3ef2bdebd1535` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.439102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f65d1049f874d27b9e3ef2bdebd1535)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.448499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.451012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f00624cad6436f9629589de0ca1962, user_id=37be0b331eec44b3a5827d99e72b3860, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.454076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.567526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 896/1768] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:41 2022] PUT /identity/v3/projects/a0f00624cad6436f9629589de0ca1962/users/37be0b331eec44b3a5827d99e72b3860/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:41.568771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.653405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f65d1049f874d27b9e3ef2bdebd1535', 'request_id': 'req-c4a064f5-d579-47a7-a267-70ef444b17fa'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:41.676843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f65d1049f874d27b9e3ef2bdebd1535', 'request_id': 'req-c4a064f5-d579-47a7-a267-70ef444b17fa'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:41.711106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25640618-dfd8-46a3-b3eb-ec07701acc07 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.752870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f65d1049f874d27b9e3ef2bdebd1535', 'request_id': 'req-c4a064f5-d579-47a7-a267-70ef444b17fa'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:41.774171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] The token cache is being invalidate because project 7f65d1049f874d27b9e3ef2bdebd1535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:41.777743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:41.781539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c4a064f5-d579-47a7-a267-70ef444b17fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:41.788620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 873/1769] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:41 2022] DELETE /identity/v3/projects/7f65d1049f874d27b9e3ef2bdebd1535 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:41.798463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52682923-daa7-4f00-a6e8-858c7a01aa1f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.803770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52682923-daa7-4f00-a6e8-858c7a01aa1f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.803770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52682923-daa7-4f00-a6e8-858c7a01aa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.856590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-52682923-daa7-4f00-a6e8-858c7a01aa1f None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:41.857842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-52682923-daa7-4f00-a6e8-858c7a01aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.990949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2011a4-9cf7-462f-aea4-49579f57c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.996448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2011a4-9cf7-462f-aea4-49579f57c44e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.996825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2011a4-9cf7-462f-aea4-49579f57c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.997194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2011a4-9cf7-462f-aea4-49579f57c44e admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.004237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2011a4-9cf7-462f-aea4-49579f57c44e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:42.012913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2011a4-9cf7-462f-aea4-49579f57c44e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.017752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2011a4-9cf7-462f-aea4-49579f57c44e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.057033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 897/1770] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:42.062800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.064665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.065471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.086976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 874/1771] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:42.088153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.088613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.089143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.115965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:42.117954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.136630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 875/1772] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:42.137481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:42.138827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] MFA Rules not processed for user `37be0b331eec44b3a5827d99e72b3860`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:42.140507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.142612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.169315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfccc4-bc70-41b5-9f92-5d92de6256a5 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:42.185642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4c602-34bb-4ee3-99a6-1480ce61283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:42.186513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4c602-34bb-4ee3-99a6-1480ce61283a admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.186799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4c602-34bb-4ee3-99a6-1480ce61283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.187070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4c602-34bb-4ee3-99a6-1480ce61283a admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.187707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4c602-34bb-4ee3-99a6-1480ce61283a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.196004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4c602-34bb-4ee3-99a6-1480ce61283a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.224529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:42.320151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 876/1773] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:42.321532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60287ae6-550c-457c-9890-26eabe3e7c01 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.322577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60287ae6-550c-457c-9890-26eabe3e7c01 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.324233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60287ae6-550c-457c-9890-26eabe3e7c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.331352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 898/1774] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:42.335860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:42.337502 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.361104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ee4a3-1abf-4648-94e0-17110249ba53 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:42.377418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-60287ae6-550c-457c-9890-26eabe3e7c01 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:42.382490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-60287ae6-550c-457c-9890-26eabe3e7c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.474378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:42.477064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.478123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.478727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.493625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.495245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.500830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 877/1775] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:42.502128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60287ae6-550c-457c-9890-26eabe3e7c01 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:42.531983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60287ae6-550c-457c-9890-26eabe3e7c01 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:42.542570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 899/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:42.543187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:42.543541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:42.544715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.545804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.546590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.547499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.548339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.553421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.596284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259430d0-5ebc-456e-9cf3-7be803990f46 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:42.609378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:42.610006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.610299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.612409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.613113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.619311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.710917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 878/1777] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/users => generated 290 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:42.711992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:42.743045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4fc66-3e71-4c2b-9145-6403c4020bc2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:42.743983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 900/1778] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/projects => generated 401 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:42.745569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:42.890544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed704c9-a9c4-4e02-a318-e7d0db97a9e1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:42.911376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:42.912651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.912651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.912651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.913461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.918456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.945365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 901/1779] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:42.950048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:42.989029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:42.989875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.990156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.990521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.991736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:42.992971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:42.996705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:42.998472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462806-7d32-495c-93b0-cccad7472621 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.017520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.018323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.021863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.023014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.024150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.027671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.042511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 879/1780] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:43.044068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.058013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9a877-e318-497a-81b6-6c5866ce2afd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.072972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.075038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.075038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.075038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.075627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.079973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.092734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 880/1781] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:43.095259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.111903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb5839-1f20-4d9c-8926-dfdc81606c58 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.119237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.119982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.121069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.121069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.121348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.123283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.184816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 902/1782] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/users => generated 307 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:43.185733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.203895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bcd16-ff98-46da-8683-424287e24192 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.214049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.216393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.216747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.217521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] PATH_INFO: `/v3/projects/839f32f55f3548b6b7f4b5ea2faeff39/users/8475a83be6ce4d62adc5187766427852/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.249964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839f32f55f3548b6b7f4b5ea2faeff39, user_id=8475a83be6ce4d62adc5187766427852, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.255068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.310728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 881/1783] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:43 2022] POST /identity/v3/projects => generated 389 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:43.312051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.382974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 903/1784] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:43 2022] PUT /identity/v3/projects/839f32f55f3548b6b7f4b5ea2faeff39/users/8475a83be6ce4d62adc5187766427852/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:43.394706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.468103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd9ab6-0ccc-4225-a18c-888b194a9387 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.516403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34646bc7-d77b-412e-a51a-1ef741a93bc0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.522887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.523719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.524094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.524424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.525064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.527249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.537359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672c6f-a238-4ffc-833a-6b056038301c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.538599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672c6f-a238-4ffc-833a-6b056038301c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.538818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672c6f-a238-4ffc-833a-6b056038301c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.539190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672c6f-a238-4ffc-833a-6b056038301c admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.539945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672c6f-a238-4ffc-833a-6b056038301c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.541646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672c6f-a238-4ffc-833a-6b056038301c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.561834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 882/1785] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:43.563805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa567901-24ec-4a35-b46b-1fc15ddb611f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.564048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa567901-24ec-4a35-b46b-1fc15ddb611f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.564782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa567901-24ec-4a35-b46b-1fc15ddb611f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.609526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 904/1786] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:43 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:43.614337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.620538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aa567901-24ec-4a35-b46b-1fc15ddb611f None None] MFA Rules not processed for user `8475a83be6ce4d62adc5187766427852`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:43.622021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aa567901-24ec-4a35-b46b-1fc15ddb611f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:43.628554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfdf1b-b942-49d8-9e4b-c6cc0d2a835a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.641509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.642649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.642951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.643220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] PATH_INFO: `/v3/projects/865ba408901a49b586653e4743c87c8d/users/82c71b8524a44ca1b69bb62a91bbbb9c/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.666284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] RBAC: Authorizing `identity:create_grant(project_id=865ba408901a49b586653e4743c87c8d, user_id=82c71b8524a44ca1b69bb62a91bbbb9c, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.669589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.743270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 905/1787] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:43 2022] PUT /identity/v3/projects/865ba408901a49b586653e4743c87c8d/users/82c71b8524a44ca1b69bb62a91bbbb9c/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:43.746236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.763322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 883/1788] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:43.764476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa567901-24ec-4a35-b46b-1fc15ddb611f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.771217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:43.793691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa567901-24ec-4a35-b46b-1fc15ddb611f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.861801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a890ea-904d-4877-9d2f-034457e6babf admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.872767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.873808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.874430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.874843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.875664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.877958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.893954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 906/1789] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:43.895235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.907419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab700da-9861-4ae1-ba8e-b6f62e53195e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.913068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4a841-0bd2-4fe4-b36d-d2c60e9383d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.913808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a841-0bd2-4fe4-b36d-d2c60e9383d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.914196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a841-0bd2-4fe4-b36d-d2c60e9383d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.914654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4a841-0bd2-4fe4-b36d-d2c60e9383d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.915533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4a841-0bd2-4fe4-b36d-d2c60e9383d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.919168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4a841-0bd2-4fe4-b36d-d2c60e9383d6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.927666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.928973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.929187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.929625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.930431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.941696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.972585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 884/1790] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:43.973965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.013451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823663a5-12c5-42e9-9121-50901583ec0c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.048304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.048616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.048866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.049132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.049777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:44.051419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:44.068129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 885/1791] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:44.069924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.114216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504327a-9d38-447f-9dc0-a8f5e82387a9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.125312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.126676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.126676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.126846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] PATH_INFO: `/v3/projects/865ba408901a49b586653e4743c87c8d/users/82c71b8524a44ca1b69bb62a91bbbb9c/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.131174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=865ba408901a49b586653e4743c87c8d, user_id=82c71b8524a44ca1b69bb62a91bbbb9c, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:44.135385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:44.153213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 886/1792] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:44 2022] PUT /identity/v3/projects/865ba408901a49b586653e4743c87c8d/users/82c71b8524a44ca1b69bb62a91bbbb9c/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:44.154164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.177928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 907/1793] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:43 2022] POST /identity/v3/projects => generated 379 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:44.180286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10a6d5-1b70-49dd-8276-4cf869bdc24e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.180681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10a6d5-1b70-49dd-8276-4cf869bdc24e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.181109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10a6d5-1b70-49dd-8276-4cf869bdc24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.218861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0d10a6d5-1b70-49dd-8276-4cf869bdc24e None None] MFA Rules not processed for user `82c71b8524a44ca1b69bb62a91bbbb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:44.220310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0d10a6d5-1b70-49dd-8276-4cf869bdc24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:44.370085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 908/1794] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:44 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:44.371102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10a6d5-1b70-49dd-8276-4cf869bdc24e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.374900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4799c-cb8a-4c62-8157-f3819d7f3744 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.492702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:44.520336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10a6d5-1b70-49dd-8276-4cf869bdc24e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.528462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.529534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.530116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.530591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.531364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:44.535912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:44.592563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.594118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.594439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.594723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] PATH_INFO: `/v3/projects/2c8d9d99397b46dab6430b8aa7bc1a33/users/2cbdccd20963495198088617073d7a58/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.630340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8d9d99397b46dab6430b8aa7bc1a33, user_id=2cbdccd20963495198088617073d7a58, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:44.638710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:44.659131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 909/1795] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:44 2022] POST /identity/v3/users => generated 296 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:44.659131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.665856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:44.719261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 887/1796] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:44 2022] PUT /identity/v3/projects/2c8d9d99397b46dab6430b8aa7bc1a33/users/2cbdccd20963495198088617073d7a58/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:44.720220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.800559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acc926-0dbc-4bcb-892a-51fd78f6335b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.808582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1b62e-457d-4662-b1b1-2ae5a31c4477 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.815988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4f942-e23d-4855-82b6-6904ba9d5534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.817810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4f942-e23d-4855-82b6-6904ba9d5534 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.818408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4f942-e23d-4855-82b6-6904ba9d5534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.819650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4f942-e23d-4855-82b6-6904ba9d5534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.820800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4f942-e23d-4855-82b6-6904ba9d5534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:44.823200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4f942-e23d-4855-82b6-6904ba9d5534 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:44.841455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 888/1797] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:44.843187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f248-8796-4980-b5c7-054db2111f2d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.843478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f248-8796-4980-b5c7-054db2111f2d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.843755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642f248-8796-4980-b5c7-054db2111f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.890073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a642f248-8796-4980-b5c7-054db2111f2d None None] MFA Rules not processed for user `2cbdccd20963495198088617073d7a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:44.893557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a642f248-8796-4980-b5c7-054db2111f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:44.926733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.929468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.930457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.930752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.995271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 889/1798] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:44.995945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:44.999832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f248-8796-4980-b5c7-054db2111f2d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.015105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642f248-8796-4980-b5c7-054db2111f2d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.020870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.021632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.021954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.022390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] PATH_INFO: `/v3/projects/9b0b19295ad747edab283e64fdef7f7a/users/c5bd05fb4f79470ba09cb8aa07bbc75a/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.024856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.026446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.042546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0b19295ad747edab283e64fdef7f7a, user_id=c5bd05fb4f79470ba09cb8aa07bbc75a, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.045418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.064127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 910/1799] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:44 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:45.143761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.148590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 890/1800] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:44 2022] PUT /identity/v3/projects/9b0b19295ad747edab283e64fdef7f7a/users/c5bd05fb4f79470ba09cb8aa07bbc75a/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:45.153546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.155014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcdeb8-ed8f-474b-9191-1b32bd7eb710 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.238131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.241737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.242268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.242636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.244831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be052b-dab7-4247-b6ba-bb821feb3bce admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.252672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.253618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.254043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.254590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.255444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.257885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.271700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 891/1801] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:45.277674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.291044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd71c9-50da-49dc-8153-8f1424cd4b36 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.296621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa85ef3-1240-460f-9422-bad3fd4921a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.297969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85ef3-1240-460f-9422-bad3fd4921a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.298946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85ef3-1240-460f-9422-bad3fd4921a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.299282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa85ef3-1240-460f-9422-bad3fd4921a8 admin admin] PATH_INFO: `/v3/projects/9b0b19295ad747edab283e64fdef7f7a/users/c5bd05fb4f79470ba09cb8aa07bbc75a/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.303668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa85ef3-1240-460f-9422-bad3fd4921a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0b19295ad747edab283e64fdef7f7a, user_id=c5bd05fb4f79470ba09cb8aa07bbc75a, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.310316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa85ef3-1240-460f-9422-bad3fd4921a8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.331382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 892/1802] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:45 2022] PUT /identity/v3/projects/9b0b19295ad747edab283e64fdef7f7a/users/c5bd05fb4f79470ba09cb8aa07bbc75a/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:45.345004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610bf94-6683-43f2-b214-f259bc3c319e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.345389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610bf94-6683-43f2-b214-f259bc3c319e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.345564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610bf94-6683-43f2-b214-f259bc3c319e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.352055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.353577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.381258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 911/1803] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:45.403719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5610bf94-6683-43f2-b214-f259bc3c319e None None] MFA Rules not processed for user `c5bd05fb4f79470ba09cb8aa07bbc75a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:45.406076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5610bf94-6683-43f2-b214-f259bc3c319e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:45.639465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 893/1804] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:45.718882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:45.865024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.867243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:45.879534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf52052-2be7-410f-a279-656d8b8c2def service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.918866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:45.958281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.959046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.960844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.961850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.976971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.980375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.997683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 912/1805] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:46.086532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610bf94-6683-43f2-b214-f259bc3c319e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:46.103334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610bf94-6683-43f2-b214-f259bc3c319e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:46.112459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:46.112459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.112459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.112459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.171767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:46.173288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:46.184871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 894/1806] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:46 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:46.503511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:46.701282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:48.713330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:48.722800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:48.732594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387aea54-c2cb-426e-b178-780935c18105 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:48.737418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5bd05fb4f79470ba09cb8aa07bbc75a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b0b19295ad747edab283e64fdef7f7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:48.737944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:48.738224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:48.738540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:48.739183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:48.739484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:48.741108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:48.753044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 913/1807] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:48.757295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:48.768406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb50c85-1493-42a4-9f21-4a08a9c6420e service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:48.772837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5bd05fb4f79470ba09cb8aa07bbc75a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b0b19295ad747edab283e64fdef7f7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:48.775992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:48.775992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:48.775992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:48.775992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:48.777416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:48.871762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 895/1808] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:48 2022] POST /identity/v3/projects => generated 355 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:48.873217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:48.883865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ec272-18d6-46f6-b4da-fad9e7fdafd2 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:48.893696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:48.947714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5bd05fb4f79470ba09cb8aa07bbc75a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b0b19295ad747edab283e64fdef7f7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:48.948342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:48.948623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:48.948898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:48.949541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:48.949845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:48.951480 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:48.963109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 914/1809] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:49.004545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:49.016418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf220f-0244-435e-b227-463ea3d67f0d tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:49.016779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:49.018653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:49.021390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5bd05fb4f79470ba09cb8aa07bbc75a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9b0b19295ad747edab283e64fdef7f7a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:49.022114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:49.022544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:49.022891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] PATH_INFO: `/v3/projects/34bda102dc3841f79d43fc133de08fba` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:49.024158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorizing `identity:delete_project(project_id=34bda102dc3841f79d43fc133de08fba)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:49.026167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:49.086217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6610bd8-a23a-4cc5-846e-c77c5628a077 tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:49.090954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:49.091601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:49.091908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:49.092178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] PATH_INFO: `/v3/users/81bf2a5a130d4204893204af69abb74c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:49.093412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81bf2a5a130d4204893204af69abb74c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:49.095261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:49.140926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34bda102dc3841f79d43fc133de08fba', 'request_id': 'req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.160464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34bda102dc3841f79d43fc133de08fba', 'request_id': 'req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.289310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34bda102dc3841f79d43fc133de08fba', 'request_id': 'req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.300397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] The token cache is being invalidate because project 34bda102dc3841f79d43fc133de08fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:49.300695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.301380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-81da0e68-d3cb-4aa7-a210-b4c08c5e0e7f tempest-VolumeQuotaClassesTest-694243512 tempest-VolumeQuotaClassesTest-694243512-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.302625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81bf2a5a130d4204893204af69abb74c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.302943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 896/1810] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:49 2022] DELETE /identity/v3/projects/34bda102dc3841f79d43fc133de08fba => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:49.380261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81bf2a5a130d4204893204af69abb74c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.393149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81bf2a5a130d4204893204af69abb74c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.393784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81bf2a5a130d4204893204af69abb74c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.394425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81bf2a5a130d4204893204af69abb74c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:49.407088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 915/1811] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:49 2022] DELETE /identity/v3/users/81bf2a5a130d4204893204af69abb74c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:49.411626 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:49.411915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:49.412189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:49.427766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:49.433058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:49.434259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:49.494711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 897/1812] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:49.504575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:49.505986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:49.529759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:49.581721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528c7d5-a612-4e62-a4e7-b93502b1234d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:49.586591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:49.587207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:49.587705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:49.587973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:49.600550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:49.602226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:49.613898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 916/1813] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:50.058108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:50.059547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:50.071519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3165d-4161-4ca8-ab28-32463b105908 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:50.076352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:50.076966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:50.077243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:50.077515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] PATH_INFO: `/v3/users/8475a83be6ce4d62adc5187766427852` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:50.078688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8475a83be6ce4d62adc5187766427852)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:50.080606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:50.132175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:50.134186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:50.149586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5beaf76-68e9-4128-8ad3-7b184f9850a9 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:50.154966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:50.155599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:50.155877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:50.156150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] PATH_INFO: `/v3/projects/d928ecfce43944ee80fa940e6f363a22` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:50.157295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d928ecfce43944ee80fa940e6f363a22)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:50.159317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:50.228711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8475a83be6ce4d62adc5187766427852'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.265527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd928ecfce43944ee80fa940e6f363a22', 'request_id': 'req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.306846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8475a83be6ce4d62adc5187766427852'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.319713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8475a83be6ce4d62adc5187766427852'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.329310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8475a83be6ce4d62adc5187766427852'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.330016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6cf010ca-ed96-48d9-921c-e65dee137bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8475a83be6ce4d62adc5187766427852'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.336239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 898/1814] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:50 2022] DELETE /identity/v3/users/8475a83be6ce4d62adc5187766427852 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:50.340624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:50.341077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:50.341358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:50.360916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:50.362314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:50.369524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd928ecfce43944ee80fa940e6f363a22', 'request_id': 'req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.376462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd928ecfce43944ee80fa940e6f363a22', 'request_id': 'req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.387771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] The token cache is being invalidate because project d928ecfce43944ee80fa940e6f363a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:50.388189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.389314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.390833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 917/1815] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:50 2022] DELETE /identity/v3/projects/d928ecfce43944ee80fa940e6f363a22 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:50.394361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:50.395850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:50.428287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 899/1816] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:50.439281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:50.444263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:50.466902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bd223-2a71-42f1-b2e2-bdc101f88b8b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:50.472945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:50.473593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:50.473892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:50.474144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] PATH_INFO: `/v3/users/abbee575ca9d42c6a1e7d3f06430c2dd` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:50.475486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbee575ca9d42c6a1e7d3f06430c2dd)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:50.478300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:50.480322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:50.506826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd0573-cae7-4ddb-8aa5-80bb4bf91b40 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:50.511815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:50.516374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:50.516374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:50.516374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:50.522883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:50.524336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:50.533821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 900/1817] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:50.697838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:50.700411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbee575ca9d42c6a1e7d3f06430c2dd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.802457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbee575ca9d42c6a1e7d3f06430c2dd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.818469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbee575ca9d42c6a1e7d3f06430c2dd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.821712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbee575ca9d42c6a1e7d3f06430c2dd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.822393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbee575ca9d42c6a1e7d3f06430c2dd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:50.838638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 918/1818] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:50 2022] DELETE /identity/v3/users/abbee575ca9d42c6a1e7d3f06430c2dd => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:51.045370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:51.047369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:51.061310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80329e86-e010-41ea-a9be-56315b763390 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:51.122589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:51.123223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:51.123501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:51.123773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] PATH_INFO: `/v3/projects/839f32f55f3548b6b7f4b5ea2faeff39` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:51.124896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=839f32f55f3548b6b7f4b5ea2faeff39)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:51.126851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:51.260256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839f32f55f3548b6b7f4b5ea2faeff39', 'request_id': 'req-63ace7e5-0d46-43d0-ae18-dc9820849947'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.266768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839f32f55f3548b6b7f4b5ea2faeff39', 'request_id': 'req-63ace7e5-0d46-43d0-ae18-dc9820849947'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.351775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839f32f55f3548b6b7f4b5ea2faeff39', 'request_id': 'req-63ace7e5-0d46-43d0-ae18-dc9820849947'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.362644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] The token cache is being invalidate because project 839f32f55f3548b6b7f4b5ea2faeff39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:51.362932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.363554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.365026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 901/1819] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:51 2022] DELETE /identity/v3/projects/839f32f55f3548b6b7f4b5ea2faeff39 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:51.369751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:51.371158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:51.384915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dbb4c-4f5b-4433-8882-bf9fcde24cc5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:51.439172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:51.439785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:51.440059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:51.440325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] PATH_INFO: `/v3/users/c5bd05fb4f79470ba09cb8aa07bbc75a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:51.441450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5bd05fb4f79470ba09cb8aa07bbc75a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:51.447618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:51.459245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:51.464822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:51.485262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ace7e5-0d46-43d0-ae18-dc9820849947 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:51.490098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:51.490759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:51.491211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:51.491484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] PATH_INFO: `/v3/projects/174eb651d83d46a6922d3a22b7ed207a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:51.493509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=174eb651d83d46a6922d3a22b7ed207a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:51.495607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:51.541732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:51.738311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174eb651d83d46a6922d3a22b7ed207a', 'request_id': 'req-531ef00f-3e2e-4afb-a14e-91b505f5dd20'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.738967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5bd05fb4f79470ba09cb8aa07bbc75a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.745429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174eb651d83d46a6922d3a22b7ed207a', 'request_id': 'req-531ef00f-3e2e-4afb-a14e-91b505f5dd20'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.808424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5bd05fb4f79470ba09cb8aa07bbc75a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.821712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5bd05fb4f79470ba09cb8aa07bbc75a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.822406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5bd05fb4f79470ba09cb8aa07bbc75a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.823018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8ae5e51e-6339-42f1-a699-107f7dd09e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5bd05fb4f79470ba09cb8aa07bbc75a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.835212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 919/1820] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:51 2022] DELETE /identity/v3/users/c5bd05fb4f79470ba09cb8aa07bbc75a => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:51.912177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174eb651d83d46a6922d3a22b7ed207a', 'request_id': 'req-531ef00f-3e2e-4afb-a14e-91b505f5dd20'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.926581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] The token cache is being invalidate because project 174eb651d83d46a6922d3a22b7ed207a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:51.927906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.928557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-531ef00f-3e2e-4afb-a14e-91b505f5dd20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:51.931410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 902/1821] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:51 2022] DELETE /identity/v3/projects/174eb651d83d46a6922d3a22b7ed207a => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:51.936454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd83f-5fa3-4834-8fcd-f7dab7869078 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:51.936744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd83f-5fa3-4834-8fcd-f7dab7869078 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:51.937012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd83f-5fa3-4834-8fcd-f7dab7869078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:51.957380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-756cd83f-5fa3-4834-8fcd-f7dab7869078 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:51.958613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-756cd83f-5fa3-4834-8fcd-f7dab7869078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:51.998009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:52.028743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 920/1822] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:52.037678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6718a-020e-48d0-9892-977d0240ef55 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.038372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6718a-020e-48d0-9892-977d0240ef55 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.041974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6718a-020e-48d0-9892-977d0240ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.064960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7cf6718a-020e-48d0-9892-977d0240ef55 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:52.066537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7cf6718a-020e-48d0-9892-977d0240ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:52.076158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 903/1823] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:52.089605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cd83f-5fa3-4834-8fcd-f7dab7869078 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.101206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cd83f-5fa3-4834-8fcd-f7dab7869078 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.119290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.119916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.120198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.120473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.121129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:52.121434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.123101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.139801 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 921/1824] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:52.143983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6718a-020e-48d0-9892-977d0240ef55 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.154148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6718a-020e-48d0-9892-977d0240ef55 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.159896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.160699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.161130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.161864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.162755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.168853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.296579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 904/1825] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:52.300689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.311094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5de2e-3438-4393-ba35-361e2eb63eb9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.325054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.328752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:52.402993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.402993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.402993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.402993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.403674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.403674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.406019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c8e2-6e15-430c-990e-636f5310ad87 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.415725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.425044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.425044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.425044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] PATH_INFO: `/v3/projects/9b0b19295ad747edab283e64fdef7f7a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.425044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0b19295ad747edab283e64fdef7f7a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.425044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.523971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 922/1826] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:52.530929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.541153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac354794-ab7f-4df7-8ca7-e17da21b9643 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.546389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.546996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.547370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.547535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.548172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.549760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.562860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 923/1827] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:52.566589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.575649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0b19295ad747edab283e64fdef7f7a', 'request_id': 'req-489ae69c-abef-4ea8-a8c4-c794c757621d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:52.578179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab10395-d908-4a18-a8fa-72ad3715e665 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.582503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0b19295ad747edab283e64fdef7f7a', 'request_id': 'req-489ae69c-abef-4ea8-a8c4-c794c757621d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:52.637900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.638534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.638947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.639224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.639864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.641470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.653382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 924/1828] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:52.657536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.660786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0b19295ad747edab283e64fdef7f7a', 'request_id': 'req-489ae69c-abef-4ea8-a8c4-c794c757621d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:52.668390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e07c2a-b0c4-443b-b4ad-3f5b30a1a102 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.673406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf7a28-afd3-420b-a104-3432067efc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.673782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] The token cache is being invalidate because project 9b0b19295ad747edab283e64fdef7f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:52.674054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf7a28-afd3-420b-a104-3432067efc7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.674204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:52.674390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf7a28-afd3-420b-a104-3432067efc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.674706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf7a28-afd3-420b-a104-3432067efc7c admin admin] PATH_INFO: `/v3/projects/c1c297b6d914487c90651bbaf18a816b/users/ddea0272cae646c8a5f130beb630739e/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.675122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-489ae69c-abef-4ea8-a8c4-c794c757621d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:52.676914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 905/1829] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:52 2022] DELETE /identity/v3/projects/9b0b19295ad747edab283e64fdef7f7a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:52.685728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4426a-d7d1-47f9-8906-e5f04d455daa None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.687187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4426a-d7d1-47f9-8906-e5f04d455daa None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.687720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4426a-d7d1-47f9-8906-e5f04d455daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.691963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf7a28-afd3-420b-a104-3432067efc7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c297b6d914487c90651bbaf18a816b, user_id=ddea0272cae646c8a5f130beb630739e, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.694898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf7a28-afd3-420b-a104-3432067efc7c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.713614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2fa4426a-d7d1-47f9-8906-e5f04d455daa None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:52.715165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2fa4426a-d7d1-47f9-8906-e5f04d455daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:52.723367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 906/1830] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:52.732170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387921f-e62d-44ab-b9a9-2bd908b9690a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.734345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387921f-e62d-44ab-b9a9-2bd908b9690a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.734345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387921f-e62d-44ab-b9a9-2bd908b9690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.742001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:52.752381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8387921f-e62d-44ab-b9a9-2bd908b9690a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:52.754211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8387921f-e62d-44ab-b9a9-2bd908b9690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:52.769728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 925/1831] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:52 2022] PUT /identity/v3/projects/c1c297b6d914487c90651bbaf18a816b/users/ddea0272cae646c8a5f130beb630739e/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:52.782136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.783621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.786864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.818821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] MFA Rules not processed for user `ddea0272cae646c8a5f130beb630739e`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:52.820194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:52.822454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 907/1832] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:52.824663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387921f-e62d-44ab-b9a9-2bd908b9690a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.844298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:52.847901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387921f-e62d-44ab-b9a9-2bd908b9690a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.853471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.854083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.854390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.854662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.855466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:52.855770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.857422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.870950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 908/1833] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:52.875232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.885074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6998989-a26f-4a4a-9d47-34e07de25db8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.890034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.890655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.890939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.891212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.891825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.893715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.906256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 926/1834] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:52.913321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:52.916417 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:52.931246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed23fd-bbbd-4bd7-a5b5-32d5aa65bb05 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:52.936044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:52.936650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:52.936930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:52.937205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:52.937832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:52.939745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:52.982217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 909/1835] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:52.986397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.011979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 927/1836] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:53.015939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.049453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdbfa6-bcbc-4113-b9c0-23b4e8f2df92 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.078847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da472ef-d63d-49a6-ac7b-3b47e3943553 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.083488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.084079 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.084354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.084620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.085483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.088487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.105864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.106494 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.106771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.107035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.107673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.109570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.199455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 928/1837] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:53 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:53.202924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.214713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd353bb9-1754-48ca-a0ba-5c72b4e2dc8d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.219203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.219820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.220096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.220362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.220999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.222600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.236682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 929/1838] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:53.242676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 910/1839] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:52 2022] POST /identity/v3/users => generated 292 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:53.242836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.243957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.250216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477f738-97fd-46b9-b593-05757889abf4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.255531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61bf2a-a98b-48d2-b518-3eef458871cc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.256621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.257273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.258156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.258523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] PATH_INFO: `/v3/projects/5009ae5f7058403888696669889d89ca/users/198d36e7c6854b2ea3e543937d7306b9/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.261482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.262193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.262781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.265628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.265628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.267407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.269158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:53.281219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 911/1840] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:53.285064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.306869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5009ae5f7058403888696669889d89ca, user_id=198d36e7c6854b2ea3e543937d7306b9, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.309582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.310159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f197b3a-5b7a-43bb-a835-82dbba5569ef admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.333256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.334031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.334679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.334679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.335556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.337852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.349074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 912/1841] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:53.353576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.363367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c2c2-a4d1-41c9-be44-c9251136f7c4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.368581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f50100-8126-4c5a-9e55-696ae053f6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.369220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f50100-8126-4c5a-9e55-696ae053f6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.369493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f50100-8126-4c5a-9e55-696ae053f6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.369760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f50100-8126-4c5a-9e55-696ae053f6fc admin admin] PATH_INFO: `/v3/projects/f5fafe6c309e40f0b2d57ed576450062/users/8a1e5ac992a7477f8d068cf76b3a2550/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.384759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f50100-8126-4c5a-9e55-696ae053f6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5fafe6c309e40f0b2d57ed576450062, user_id=8a1e5ac992a7477f8d068cf76b3a2550, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.387429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f50100-8126-4c5a-9e55-696ae053f6fc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.409502 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 930/1842] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:53 2022] PUT /identity/v3/projects/5009ae5f7058403888696669889d89ca/users/198d36e7c6854b2ea3e543937d7306b9/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:53.413252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.456876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:53.477239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5e1fb-9dd8-48e0-9a03-a27abeb2ca63 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.482498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.483073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.483415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.483850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.484569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.486154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.492413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 913/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:53 2022] PUT /identity/v3/projects/f5fafe6c309e40f0b2d57ed576450062/users/8a1e5ac992a7477f8d068cf76b3a2550/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:53.498262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 931/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:53.501947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.502406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043fd33-d317-418a-a049-7f52c27c1c81 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.502693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043fd33-d317-418a-a049-7f52c27c1c81 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.502964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043fd33-d317-418a-a049-7f52c27c1c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.514142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400aec67-eec8-4a0c-94b6-d8bb870729aa admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.533690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8043fd33-d317-418a-a049-7f52c27c1c81 None None] MFA Rules not processed for user `8a1e5ac992a7477f8d068cf76b3a2550`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:53.535434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8043fd33-d317-418a-a049-7f52c27c1c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:53.581614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8d53f-34fa-452c-a74e-4109cdcbb6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.582467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8d53f-34fa-452c-a74e-4109cdcbb6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.582814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8d53f-34fa-452c-a74e-4109cdcbb6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.583164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8d53f-34fa-452c-a74e-4109cdcbb6c6 admin admin] PATH_INFO: `/v3/projects/5009ae5f7058403888696669889d89ca/users/198d36e7c6854b2ea3e543937d7306b9/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.587004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8d53f-34fa-452c-a74e-4109cdcbb6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5009ae5f7058403888696669889d89ca, user_id=198d36e7c6854b2ea3e543937d7306b9, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.590421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8d53f-34fa-452c-a74e-4109cdcbb6c6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.601181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 932/1845] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:53 2022] PUT /identity/v3/projects/5009ae5f7058403888696669889d89ca/users/198d36e7c6854b2ea3e543937d7306b9/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:53.612620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.613702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.613984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.624060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 914/1846] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:53.631360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043fd33-d317-418a-a049-7f52c27c1c81 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.646424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043fd33-d317-418a-a049-7f52c27c1c81 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.647947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] MFA Rules not processed for user `198d36e7c6854b2ea3e543937d7306b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:53.649181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:53.706266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.706906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.707327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.707519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.708128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.710869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.723085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 933/1847] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:53.736278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.737902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:53.797455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7893-b4f3-4b16-9f1c-1af8fc5cdb92 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.801307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.801912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.802199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.802499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.813739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.817923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:53.818300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 915/1848] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:53 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:53.822457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:53.822972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:53.830173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 934/1849] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:53.868659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:53.892982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ccad8-fecd-4aef-9a46-e28fa97dc56a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:53.898093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:53.899542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:53.900126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:53.900411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:53.901349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:53.903508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:54.016279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 916/1850] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:53 2022] POST /identity/v3/users => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:54.020409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:54.031633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1b259-79be-40c7-b214-9884e31eb45e service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:54.037687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:54.039717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:54.040309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:54.040686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:54.041472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:54.043572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:54.059112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 935/1851] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:54.063629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:54.074671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3280ffd-321b-4750-9ade-b2d2369d9c78 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:54.079421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:54.080106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:54.080529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:54.080704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] PATH_INFO: `/v3/projects/8f8f5d83c12d4797a7512d23fef7c27f/users/c66c54f0b7af46149ea26b7f4e666a29/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:54.098546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8f5d83c12d4797a7512d23fef7c27f, user_id=c66c54f0b7af46149ea26b7f4e666a29, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:54.101851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:54.215889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 917/1852] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:54 2022] PUT /identity/v3/projects/8f8f5d83c12d4797a7512d23fef7c27f/users/c66c54f0b7af46149ea26b7f4e666a29/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:54.224966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:54.231890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17323b4-f695-4897-98c0-166da9b0e750 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:54.312479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d74162-ebeb-4dd6-a67b-f3871e2b6ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:54.313110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74162-ebeb-4dd6-a67b-f3871e2b6ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:54.313420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74162-ebeb-4dd6-a67b-f3871e2b6ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:54.313688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d74162-ebeb-4dd6-a67b-f3871e2b6ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:54.314381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d74162-ebeb-4dd6-a67b-f3871e2b6ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:54.316374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d74162-ebeb-4dd6-a67b-f3871e2b6ddb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:54.327193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 936/1853] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:54.331663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:54.354521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164c5a5-c127-4392-a314-2747825b7b1b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:54.368711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:54.369656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:54.370172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:54.371357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] PATH_INFO: `/v3/projects/8f8f5d83c12d4797a7512d23fef7c27f/users/c66c54f0b7af46149ea26b7f4e666a29/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:54.375084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8f5d83c12d4797a7512d23fef7c27f, user_id=c66c54f0b7af46149ea26b7f4e666a29, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:54.379569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:54.395511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 918/1854] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:54 2022] PUT /identity/v3/projects/8f8f5d83c12d4797a7512d23fef7c27f/users/c66c54f0b7af46149ea26b7f4e666a29/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:54.408446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55abc5-1637-4611-af2e-15e33108fed6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:54.408804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55abc5-1637-4611-af2e-15e33108fed6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:54.409142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55abc5-1637-4611-af2e-15e33108fed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:54.458690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4f55abc5-1637-4611-af2e-15e33108fed6 None None] MFA Rules not processed for user `c66c54f0b7af46149ea26b7f4e666a29`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:54.458690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4f55abc5-1637-4611-af2e-15e33108fed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:54.572352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 937/1855] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:54.668548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:54.670215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:54.729827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790f79-812f-44ce-92ce-844e2d9e2b83 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:54.733395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:54.733993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:54.734270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:54.734795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:54.744599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:54.745965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:54.755783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 919/1856] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:54.979013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55abc5-1637-4611-af2e-15e33108fed6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:54.988761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55abc5-1637-4611-af2e-15e33108fed6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:54.992927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92ced8-ce38-4587-ac82-68095e2b04b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:55.002316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92ced8-ce38-4587-ac82-68095e2b04b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:55.002316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92ced8-ce38-4587-ac82-68095e2b04b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:55.002316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92ced8-ce38-4587-ac82-68095e2b04b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:55.051655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92ced8-ce38-4587-ac82-68095e2b04b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:55.059824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92ced8-ce38-4587-ac82-68095e2b04b0 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:55.063497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 938/1857] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:55.774912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:55.777286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:55.857736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15a81-e3df-4e3f-ab89-cec8409df6b9 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:55.862281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:55.862984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:55.863263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:55.863535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] PATH_INFO: `/v3/users/37be0b331eec44b3a5827d99e72b3860` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:55.864692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37be0b331eec44b3a5827d99e72b3860)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:55.866680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:56.120995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:56.134086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37be0b331eec44b3a5827d99e72b3860'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:56.191655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37be0b331eec44b3a5827d99e72b3860'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:56.204230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37be0b331eec44b3a5827d99e72b3860'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:56.211704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37be0b331eec44b3a5827d99e72b3860'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:56.212336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37be0b331eec44b3a5827d99e72b3860'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:56.217977 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 920/1858] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:55 2022] DELETE /identity/v3/users/37be0b331eec44b3a5827d99e72b3860 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:56.222599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:56.223065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:56.223152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:56.236106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:56.242132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:56.244873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:56.319067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 939/1859] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:56.327763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:56.329554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:56.389202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd984fd-dee4-43ec-b8ba-6bfa945f17f0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:56.392866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:56.393474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:56.393752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:56.394021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:56.403377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:56.404812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:56.415649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 921/1860] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:56.833096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:56.834596 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:56.851662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f3503-dd30-4b6d-8e14-e72e5025e2be None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:56.856792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:56.857664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:56.858233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:56.858705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] PATH_INFO: `/v3/projects/a0f00624cad6436f9629589de0ca1962` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:56.859866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f00624cad6436f9629589de0ca1962)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:56.861816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:57.063214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f00624cad6436f9629589de0ca1962', 'request_id': 'req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.167398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f00624cad6436f9629589de0ca1962', 'request_id': 'req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.174338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f00624cad6436f9629589de0ca1962', 'request_id': 'req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.184641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] The token cache is being invalidate because project a0f00624cad6436f9629589de0ca1962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:57.184928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.185562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.187110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 940/1861] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:56 2022] DELETE /identity/v3/projects/a0f00624cad6436f9629589de0ca1962 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:57.191120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:57.205237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955cfa0-f81b-45d3-a4c7-643565850a92 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:57.262641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:57.263281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:57.263556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:57.263828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] PATH_INFO: `/v3/users/82c71b8524a44ca1b69bb62a91bbbb9c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:57.264967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c71b8524a44ca1b69bb62a91bbbb9c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:57.266944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:57.461420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c71b8524a44ca1b69bb62a91bbbb9c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.528757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c71b8524a44ca1b69bb62a91bbbb9c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.530341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:57.548697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c71b8524a44ca1b69bb62a91bbbb9c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.559090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c71b8524a44ca1b69bb62a91bbbb9c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.560018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c71b8524a44ca1b69bb62a91bbbb9c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:57.569256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 922/1862] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:57 2022] DELETE /identity/v3/users/82c71b8524a44ca1b69bb62a91bbbb9c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:57.763985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:57.765530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:57.780396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd97f-0a99-4a4c-8f1f-0841da139d34 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:57.838693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:57.839319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:57.839607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:57.839858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] PATH_INFO: `/v3/users/ddea0272cae646c8a5f130beb630739e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:57.841008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddea0272cae646c8a5f130beb630739e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:57.842969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.044065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.045961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:58.062314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2521b6-6813-4c98-9562-63f769be3056 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.063765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:58.070493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.071790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.072148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.072487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] PATH_INFO: `/v3/projects/865ba408901a49b586653e4743c87c8d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.073951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=865ba408901a49b586653e4743c87c8d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.076041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.084978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddea0272cae646c8a5f130beb630739e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.173895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddea0272cae646c8a5f130beb630739e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.193277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865ba408901a49b586653e4743c87c8d', 'request_id': 'req-5e1b704e-40eb-4ff3-a093-baf173e758e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.194011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddea0272cae646c8a5f130beb630739e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.196585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddea0272cae646c8a5f130beb630739e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.197249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0e1c7a36-75a9-4307-9f14-66123ee75db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddea0272cae646c8a5f130beb630739e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.201304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865ba408901a49b586653e4743c87c8d', 'request_id': 'req-5e1b704e-40eb-4ff3-a093-baf173e758e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.216252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 941/1863] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:57 2022] DELETE /identity/v3/users/ddea0272cae646c8a5f130beb630739e => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:58.220989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.221353 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.221704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.248489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:58.249796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:58.300805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865ba408901a49b586653e4743c87c8d', 'request_id': 'req-5e1b704e-40eb-4ff3-a093-baf173e758e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.310470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 942/1864] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:58.315506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] The token cache is being invalidate because project 865ba408901a49b586653e4743c87c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:58.315798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.316452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5e1b704e-40eb-4ff3-a093-baf173e758e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:58.318084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 923/1865] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:58 2022] DELETE /identity/v3/projects/865ba408901a49b586653e4743c87c8d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:58.320115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.321735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:58.322738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3377-9392-4245-8197-ddff3053dede None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.323022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3377-9392-4245-8197-ddff3053dede None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.323297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce3377-9392-4245-8197-ddff3053dede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.343447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f7ce3377-9392-4245-8197-ddff3053dede None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:58.344705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f7ce3377-9392-4245-8197-ddff3053dede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:58.354122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 924/1866] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:58.361562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934cd45-cd67-46e2-8972-750a2ba4e00a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.362020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934cd45-cd67-46e2-8972-750a2ba4e00a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.362291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934cd45-cd67-46e2-8972-750a2ba4e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.382829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8934cd45-cd67-46e2-8972-750a2ba4e00a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:58.384102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8934cd45-cd67-46e2-8972-750a2ba4e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:58.384845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a169108-9f91-4b89-825c-76cdae6a3fbd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.388475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.389099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.389382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.389656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.391364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:58.391901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 925/1867] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:58.395997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934cd45-cd67-46e2-8972-750a2ba4e00a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.403838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.405260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.405369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934cd45-cd67-46e2-8972-750a2ba4e00a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.410063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.410677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.411071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.411349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.411988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:58.412288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.414051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.414977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 943/1868] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:58.425972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 926/1869] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:32:58.431201 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.440670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a263f32-efa6-403d-9b0d-66880ed082ad service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.446137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.446918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.447310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.447587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.448213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.450113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.547509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 944/1870] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:58 2022] POST /identity/v3/projects => generated 407 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:58.551929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.563340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31e24b-f8d3-4e86-8b0d-55903a7183b4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.622725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.623455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.623733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.624247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.624738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.626762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.801260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 927/1871] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:58 2022] POST /identity/v3/users => generated 304 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:58.804910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.815349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9488de9-02c2-4d80-bba7-d360a6fc8f71 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.820467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.820805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.821082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.824284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.824284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.824996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.836591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 945/1872] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:58.840487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.851390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405ae56-e730-40f5-ae10-d547c7df1d91 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.856438 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.856901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.857182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.857458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.858097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.859789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.874051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 928/1873] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:58.878428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.888947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ce614-2c43-40ea-91ad-9d9d15138119 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.893467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a75dda-59b5-4250-9687-114e4ad3373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.894116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75dda-59b5-4250-9687-114e4ad3373b admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.894395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75dda-59b5-4250-9687-114e4ad3373b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.894665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a75dda-59b5-4250-9687-114e4ad3373b admin admin] PATH_INFO: `/v3/projects/982ec12120f942de98a05f4af3534b85/users/f558e73b196d46229f87873b8ecb73ac/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.909487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a75dda-59b5-4250-9687-114e4ad3373b admin admin] RBAC: Authorizing `identity:create_grant(project_id=982ec12120f942de98a05f4af3534b85, user_id=f558e73b196d46229f87873b8ecb73ac, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:58.912129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a75dda-59b5-4250-9687-114e4ad3373b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:58.915506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:58.970553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:58.972305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:58.988120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b6089-aa64-44a9-a690-cd7ce556ab57 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:58.995933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:58.995933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:58.995933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:58.996375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] PATH_INFO: `/v3/projects/c1c297b6d914487c90651bbaf18a816b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:58.997357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c297b6d914487c90651bbaf18a816b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.000088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.002304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:59.019609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 946/1874] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:58 2022] PUT /identity/v3/projects/982ec12120f942de98a05f4af3534b85/users/f558e73b196d46229f87873b8ecb73ac/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:59.031588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067127e-a5e6-4a9d-b012-0d839174f344 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.031860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067127e-a5e6-4a9d-b012-0d839174f344 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.032141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067127e-a5e6-4a9d-b012-0d839174f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.065749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b067127e-a5e6-4a9d-b012-0d839174f344 None None] MFA Rules not processed for user `f558e73b196d46229f87873b8ecb73ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:59.067758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b067127e-a5e6-4a9d-b012-0d839174f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:59.131546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 947/1875] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:59 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:59.136754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067127e-a5e6-4a9d-b012-0d839174f344 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.144862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c297b6d914487c90651bbaf18a816b', 'request_id': 'req-9615441c-1d5c-4e7b-aca8-26285a53bbcb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.151541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c297b6d914487c90651bbaf18a816b', 'request_id': 'req-9615441c-1d5c-4e7b-aca8-26285a53bbcb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.197472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067127e-a5e6-4a9d-b012-0d839174f344 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.244151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c297b6d914487c90651bbaf18a816b', 'request_id': 'req-9615441c-1d5c-4e7b-aca8-26285a53bbcb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.253328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.254387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.255039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.260199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.260199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.260373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] The token cache is being invalidate because project c1c297b6d914487c90651bbaf18a816b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:59.260373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.260373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.260573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.265274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 929/1876] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:58 2022] DELETE /identity/v3/projects/c1c297b6d914487c90651bbaf18a816b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:59.266281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.267724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:59.282004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615441c-1d5c-4e7b-aca8-26285a53bbcb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.286929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.287555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.287831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.288102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] PATH_INFO: `/v3/users/198d36e7c6854b2ea3e543937d7306b9` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.289469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=198d36e7c6854b2ea3e543937d7306b9)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.291437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.345466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 948/1877] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:59 2022] POST /identity/v3/projects => generated 407 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:59.349802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.351541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:59.375038 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:59.435875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ba68-b007-4333-8a44-c8977b8929a4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.440904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.441584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.441867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.442139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.442832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.445297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.491214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198d36e7c6854b2ea3e543937d7306b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.574988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 949/1878] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:59 2022] POST /identity/v3/users => generated 310 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:59.578726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.591983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198d36e7c6854b2ea3e543937d7306b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.593260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e8c2b-d636-4540-bcd4-3cf782a0f7dc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.598244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.598866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.599143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.599413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.600054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.601808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.605077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198d36e7c6854b2ea3e543937d7306b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.611691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 950/1879] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:59.615572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198d36e7c6854b2ea3e543937d7306b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.616214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.616320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198d36e7c6854b2ea3e543937d7306b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:59.619565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:59.623434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 930/1880] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:59 2022] DELETE /identity/v3/users/198d36e7c6854b2ea3e543937d7306b9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:59.637166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27e11b-2fc9-4a74-a53b-f681dade27d7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.697268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.697931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.698209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.698578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] PATH_INFO: `/v3/projects/4872eba106ec4396997fa938cd410a41/users/9a7fb44cb777405dbf93dee163c7a621/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.715281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4872eba106ec4396997fa938cd410a41, user_id=9a7fb44cb777405dbf93dee163c7a621, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.717975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.773668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 951/1881] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:59 2022] PUT /identity/v3/projects/4872eba106ec4396997fa938cd410a41/users/9a7fb44cb777405dbf93dee163c7a621/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:59.777638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.788765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aebd1f-6e86-441b-bc83-a6d5793c1786 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.867349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434bca8-b7ad-4fe3-a915-6321b41d2ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.867967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bca8-b7ad-4fe3-a915-6321b41d2ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.868244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bca8-b7ad-4fe3-a915-6321b41d2ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.868514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bca8-b7ad-4fe3-a915-6321b41d2ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.869157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434bca8-b7ad-4fe3-a915-6321b41d2ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.870872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434bca8-b7ad-4fe3-a915-6321b41d2ac5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.873079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:59.882195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 931/1882] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:59.887076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.897824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605289-7a06-4fd7-9e15-2aefef99f1aa admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.903106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.903787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.904083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.904452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] PATH_INFO: `/v3/projects/4872eba106ec4396997fa938cd410a41/users/9a7fb44cb777405dbf93dee163c7a621/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.906433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4872eba106ec4396997fa938cd410a41, user_id=9a7fb44cb777405dbf93dee163c7a621, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.909064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.930125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 952/1883] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:59 2022] PUT /identity/v3/projects/4872eba106ec4396997fa938cd410a41/users/9a7fb44cb777405dbf93dee163c7a621/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:59.935100 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.936071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.936480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.968688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] MFA Rules not processed for user `9a7fb44cb777405dbf93dee163c7a621`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:59.969914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.050621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 932/1884] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:32:59 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:00.062345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.064235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.111956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.114085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.139423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79853-3658-41d4-9ac4-9eaa306fc1d9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.143178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e564af-a4da-45d7-b3c9-ae02941ffb73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.143869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e564af-a4da-45d7-b3c9-ae02941ffb73 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.144212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e564af-a4da-45d7-b3c9-ae02941ffb73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.144551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e564af-a4da-45d7-b3c9-ae02941ffb73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.155627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e564af-a4da-45d7-b3c9-ae02941ffb73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.157076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e564af-a4da-45d7-b3c9-ae02941ffb73 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.168181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 953/1885] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:00.191901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456056ae-2617-4f59-b3a3-e0060a20bd0f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.196916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.197584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.197865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.198130 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] PATH_INFO: `/v3/projects/5009ae5f7058403888696669889d89ca` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.199415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5009ae5f7058403888696669889d89ca)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.201337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.328186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5009ae5f7058403888696669889d89ca', 'request_id': 'req-0bca0e7a-ca8b-425d-a95a-6942a3b95494'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.334291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5009ae5f7058403888696669889d89ca', 'request_id': 'req-0bca0e7a-ca8b-425d-a95a-6942a3b95494'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.424863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5009ae5f7058403888696669889d89ca', 'request_id': 'req-0bca0e7a-ca8b-425d-a95a-6942a3b95494'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.437995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] The token cache is being invalidate because project 5009ae5f7058403888696669889d89ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:00.438326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.438989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0bca0e7a-ca8b-425d-a95a-6942a3b95494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.440752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 933/1886] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:00 2022] DELETE /identity/v3/projects/5009ae5f7058403888696669889d89ca => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:00.446748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.447034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.447335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.468215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:00.469405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.473648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:00.541107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 954/1887] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:00.542907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ebb1c-8fa5-4fd7-8cc9-05a0cf29c5ab None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.543188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ebb1c-8fa5-4fd7-8cc9-05a0cf29c5ab None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.543456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ebb1c-8fa5-4fd7-8cc9-05a0cf29c5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.573128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.574555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.586271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-1b2ebb1c-8fa5-4fd7-8cc9-05a0cf29c5ab None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:00.589028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1b2ebb1c-8fa5-4fd7-8cc9-05a0cf29c5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.600277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 934/1888] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:00.613925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ebb1c-8fa5-4fd7-8cc9-05a0cf29c5ab None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.613925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ebb1c-8fa5-4fd7-8cc9-05a0cf29c5ab None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.619500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.620104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.620382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.620652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.621296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:00.621595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.623299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.626525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:00.639713 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638a140-5d1f-4089-98bb-654487e84bcf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.643109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 935/1889] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:00.644395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.646276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.647056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.647398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.647764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.649978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.658511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22f87a-8ca2-4f73-804b-ca78d0af70cf admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.664071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.664807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.665112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.665388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.666068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.668467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.719603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.721026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.729793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 955/1890] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:00 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:00.775762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 936/1891] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:00 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:00.779790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.790718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed381885-73ba-4eaf-bd54-2b4f8d0c7ae6 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.850295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.850891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.851171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.851442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.852064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.854025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.865083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:00.929972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 956/1892] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:00 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:00.934497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.945535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bf74f-12df-4fcc-9c14-2e46dd760959 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.950769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.951364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.951642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.952134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.952773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.954380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.965567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 937/1893] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:00.969577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.980159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb277cdc-5e64-4dca-8144-10f6ed197576 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.984771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ab581-8744-4d78-8c8f-f8143fa92b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.985547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ab581-8744-4d78-8c8f-f8143fa92b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.985986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ab581-8744-4d78-8c8f-f8143fa92b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.986524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ab581-8744-4d78-8c8f-f8143fa92b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.987155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ab581-8744-4d78-8c8f-f8143fa92b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.988742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ab581-8744-4d78-8c8f-f8143fa92b1e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.999185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 957/1894] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:01.003199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.013238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17f1cb-d896-415a-8335-21574b47cf0e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.017918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.018593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.018882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.019150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] PATH_INFO: `/v3/projects/a69bcd133a2b42309e71a5b96ee99add/users/246b8f62c4b94024b9f6b393b15ce67b/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.034525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69bcd133a2b42309e71a5b96ee99add, user_id=246b8f62c4b94024b9f6b393b15ce67b, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.037352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.113340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 938/1895] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:01 2022] PUT /identity/v3/projects/a69bcd133a2b42309e71a5b96ee99add/users/246b8f62c4b94024b9f6b393b15ce67b/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:01.124863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d843c-6209-483d-be56-7b58ae8fd1e9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.125149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d843c-6209-483d-be56-7b58ae8fd1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.125425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d843c-6209-483d-be56-7b58ae8fd1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.165291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-765d843c-6209-483d-be56-7b58ae8fd1e9 None None] MFA Rules not processed for user `246b8f62c4b94024b9f6b393b15ce67b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:01.166562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-765d843c-6209-483d-be56-7b58ae8fd1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:01.239438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 958/1896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:01.244553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.309759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073be949-5155-4e69-9472-e171a9aa6d2b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.314444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.315045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.315318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.315684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.316232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.318121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.412991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 939/1897] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:01 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:01.415020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d843c-6209-483d-be56-7b58ae8fd1e9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.425685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d843c-6209-483d-be56-7b58ae8fd1e9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.483534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.484180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.484478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.484750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.485401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.487341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.603507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 959/1898] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:01 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:01.607530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.618779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79bbe5-4a84-4487-bd09-d9cf1f0e1fd7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.623514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.624111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.624841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.624841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.625269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.626883 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.638821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 940/1899] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:01.643015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.653469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5917-50a2-46f2-b821-38872c653b31 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.658063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.658762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.659023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.659287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] PATH_INFO: `/v3/projects/701e09d0a5184a6a9cc85f6e6d225c3b/users/12b8f31f6da44e71adf8a5906e1d96dc/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.675725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701e09d0a5184a6a9cc85f6e6d225c3b, user_id=12b8f31f6da44e71adf8a5906e1d96dc, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.678433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.752816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 960/1900] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:01 2022] PUT /identity/v3/projects/701e09d0a5184a6a9cc85f6e6d225c3b/users/12b8f31f6da44e71adf8a5906e1d96dc/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:01.758533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.782270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461b50-595b-4969-864d-cc01b658234d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.849953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de93484-e706-4584-93be-4972cdf8e799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.850620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de93484-e706-4584-93be-4972cdf8e799 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.851861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de93484-e706-4584-93be-4972cdf8e799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.852139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de93484-e706-4584-93be-4972cdf8e799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.852777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de93484-e706-4584-93be-4972cdf8e799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.855644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de93484-e706-4584-93be-4972cdf8e799 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.865902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 941/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:01.871070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.881450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4f232-ffb9-4971-b3b9-fb542b14ae66 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.886311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.886981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.887254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.887520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] PATH_INFO: `/v3/projects/701e09d0a5184a6a9cc85f6e6d225c3b/users/12b8f31f6da44e71adf8a5906e1d96dc/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.889460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701e09d0a5184a6a9cc85f6e6d225c3b, user_id=12b8f31f6da44e71adf8a5906e1d96dc, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.892085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.903123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 961/1902] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:01 2022] PUT /identity/v3/projects/701e09d0a5184a6a9cc85f6e6d225c3b/users/12b8f31f6da44e71adf8a5906e1d96dc/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:01.914730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca0e8-8f18-4c10-b0de-097041580a6f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.915092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca0e8-8f18-4c10-b0de-097041580a6f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.915478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca0e8-8f18-4c10-b0de-097041580a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.930518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:01.944667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-de3ca0e8-8f18-4c10-b0de-097041580a6f None None] MFA Rules not processed for user `12b8f31f6da44e71adf8a5906e1d96dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:01.946035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-de3ca0e8-8f18-4c10-b0de-097041580a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:02.013439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 942/1903] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:02.024271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:02.026029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:02.054472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:02.083402 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535d2ce-214f-406a-8c40-412c3a8ed029 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:02.088661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa577a9-2615-4cb6-938d-2bf2847f81c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:02.089257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa577a9-2615-4cb6-938d-2bf2847f81c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:02.089537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa577a9-2615-4cb6-938d-2bf2847f81c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:02.089808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa577a9-2615-4cb6-938d-2bf2847f81c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:02.146777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa577a9-2615-4cb6-938d-2bf2847f81c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:02.148732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa577a9-2615-4cb6-938d-2bf2847f81c8 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:02.157605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 962/1904] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:12.042799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=31 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:12.153636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=37 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:17.031993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c14f2-b800-40b5-b496-d0ec238dfa19 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.032520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c14f2-b800-40b5-b496-d0ec238dfa19 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.032520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c14f2-b800-40b5-b496-d0ec238dfa19 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.034071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 943/1905] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:33:17.045658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.046141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.046690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.067260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:17.068488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.138712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 963/1906] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:33:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:17.154984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c14f2-b800-40b5-b496-d0ec238dfa19 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:17.156821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-943c14f2-b800-40b5-b496-d0ec238dfa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.180765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c14f2-b800-40b5-b496-d0ec238dfa19 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:17.187482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d06b0d-e105-40a8-996d-9cea0d2daa79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:17.188094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d06b0d-e105-40a8-996d-9cea0d2daa79 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.188376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d06b0d-e105-40a8-996d-9cea0d2daa79 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.188651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d06b0d-e105-40a8-996d-9cea0d2daa79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.198527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d06b0d-e105-40a8-996d-9cea0d2daa79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:17.200279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d06b0d-e105-40a8-996d-9cea0d2daa79 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:17.211200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 944/1907] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:17.464570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:17.466003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.532551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aedc5f-73df-4871-84d1-63b6d3fa175f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:17.537219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:17.537845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.538123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.538423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] PATH_INFO: `/v3/users/8a1e5ac992a7477f8d068cf76b3a2550` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.539581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a1e5ac992a7477f8d068cf76b3a2550)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:17.541531 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:17.703854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a1e5ac992a7477f8d068cf76b3a2550'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.779758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a1e5ac992a7477f8d068cf76b3a2550'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.793080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a1e5ac992a7477f8d068cf76b3a2550'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.793733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a1e5ac992a7477f8d068cf76b3a2550'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.794346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a1e5ac992a7477f8d068cf76b3a2550'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.807504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 964/1908] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:17 2022] DELETE /identity/v3/users/8a1e5ac992a7477f8d068cf76b3a2550 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:17.811942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea641af-1124-493a-98a7-a73b333646df None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.812228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea641af-1124-493a-98a7-a73b333646df None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.812498 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea641af-1124-493a-98a7-a73b333646df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.832741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3ea641af-1124-493a-98a7-a73b333646df None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:17.834190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3ea641af-1124-493a-98a7-a73b333646df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.891469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 945/1909] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:17.901063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:17.902498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.915840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:17.962928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab868732-c7d8-4400-b8de-59ff1e05291d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:17.966425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:17.967084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.967357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.967624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.976982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:17.978554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:17.988044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 965/1910] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:18.404083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea641af-1124-493a-98a7-a73b333646df None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.407950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3ea641af-1124-493a-98a7-a73b333646df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:18.419056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea641af-1124-493a-98a7-a73b333646df None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.423813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.424705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.425176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.425634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] PATH_INFO: `/v3/projects/f5fafe6c309e40f0b2d57ed576450062` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.426981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5fafe6c309e40f0b2d57ed576450062)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.428897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.530829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5fafe6c309e40f0b2d57ed576450062', 'request_id': 'req-484a33b6-0de3-49d5-b802-db84d81ce147'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.537603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5fafe6c309e40f0b2d57ed576450062', 'request_id': 'req-484a33b6-0de3-49d5-b802-db84d81ce147'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.705682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5fafe6c309e40f0b2d57ed576450062', 'request_id': 'req-484a33b6-0de3-49d5-b802-db84d81ce147'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.719594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] The token cache is being invalidate because project f5fafe6c309e40f0b2d57ed576450062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:18.719738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.720395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.722061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 946/1911] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:18 2022] DELETE /identity/v3/projects/f5fafe6c309e40f0b2d57ed576450062 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:18.725783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.748794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb021e5-dd43-4b28-8d70-bddc4fa67cee service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.807385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.808006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.808282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.808553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] PATH_INFO: `/v3/users/c66c54f0b7af46149ea26b7f4e666a29` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.809664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66c54f0b7af46149ea26b7f4e666a29)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.811614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.982625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66c54f0b7af46149ea26b7f4e666a29'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.047072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66c54f0b7af46149ea26b7f4e666a29'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.059863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66c54f0b7af46149ea26b7f4e666a29'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.060519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66c54f0b7af46149ea26b7f4e666a29'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.061225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ce7fba73-d90e-42a3-af7b-db76e8257519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66c54f0b7af46149ea26b7f4e666a29'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.074575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 966/1912] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:18 2022] DELETE /identity/v3/users/c66c54f0b7af46149ea26b7f4e666a29 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:19.552644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:19.554830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.568403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a33b6-0de3-49d5-b802-db84d81ce147 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:19.631691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:19.632327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.632606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.632901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] PATH_INFO: `/v3/projects/8f8f5d83c12d4797a7512d23fef7c27f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.634078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f8f5d83c12d4797a7512d23fef7c27f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:19.636140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:19.647791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:19.782827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f8f5d83c12d4797a7512d23fef7c27f', 'request_id': 'req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.789385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f8f5d83c12d4797a7512d23fef7c27f', 'request_id': 'req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.886956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f8f5d83c12d4797a7512d23fef7c27f', 'request_id': 'req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.895677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] The token cache is being invalidate because project 8f8f5d83c12d4797a7512d23fef7c27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:19.895966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.896603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f36faaed-82c6-4be8-93b7-e23f6e69b3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.898288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 947/1913] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:19 2022] DELETE /identity/v3/projects/8f8f5d83c12d4797a7512d23fef7c27f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:19.903206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6eb8d-ee04-4dbe-a85c-b5a5ee56e9cc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.903699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6eb8d-ee04-4dbe-a85c-b5a5ee56e9cc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.903872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6eb8d-ee04-4dbe-a85c-b5a5ee56e9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.924440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-64a6eb8d-ee04-4dbe-a85c-b5a5ee56e9cc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:19.925640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-64a6eb8d-ee04-4dbe-a85c-b5a5ee56e9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.957322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:19.988283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 967/1914] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:19.996291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143641b-a17b-477c-976c-1c76b290b491 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.998588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143641b-a17b-477c-976c-1c76b290b491 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.998588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143641b-a17b-477c-976c-1c76b290b491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.016656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8143641b-a17b-477c-976c-1c76b290b491 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:20.017868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8143641b-a17b-477c-976c-1c76b290b491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:20.026370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 948/1915] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:20.030697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6eb8d-ee04-4dbe-a85c-b5a5ee56e9cc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.041060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6eb8d-ee04-4dbe-a85c-b5a5ee56e9cc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.045853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.046509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.046803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.047056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.047699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:20.048001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.049621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.061818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 968/1916] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:20.066480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143641b-a17b-477c-976c-1c76b290b491 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.076512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143641b-a17b-477c-976c-1c76b290b491 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.081255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.084761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.084761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.084761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.084761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.085420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.174442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 949/1917] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:20.178297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.188369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5f50d-9fda-4b52-930a-7c94f0782baa admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.245832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.246458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.246848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.247127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.247756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.249732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.345398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 969/1918] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:20.349546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.360258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8717487-41d9-4cdf-9869-0604749e15fe admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.364901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.365518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.365794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.366064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.366733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.368379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.378560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 950/1919] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:20.382436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.395917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba330-0bcd-4e56-b46e-6037c4efc6a0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.401610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa4f8f-85ba-4b44-be51-cea76061642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.402189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa4f8f-85ba-4b44-be51-cea76061642c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.402503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa4f8f-85ba-4b44-be51-cea76061642c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.402833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa4f8f-85ba-4b44-be51-cea76061642c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.403406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa4f8f-85ba-4b44-be51-cea76061642c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.404991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa4f8f-85ba-4b44-be51-cea76061642c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.416161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 970/1920] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:20.420382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.430311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3cd5b-67e7-4f50-94ae-13b314caba15 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.435047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.435677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.435953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.436219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] PATH_INFO: `/v3/projects/65487b705788497385ca4ddae01cc5e1/users/5e2a56d31d6b4519a45467748fc38c8b/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.451610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65487b705788497385ca4ddae01cc5e1, user_id=5e2a56d31d6b4519a45467748fc38c8b, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.454246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.532740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 951/1921] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:20 2022] PUT /identity/v3/projects/65487b705788497385ca4ddae01cc5e1/users/5e2a56d31d6b4519a45467748fc38c8b/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:20.541341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5878a-bbca-422a-b7fc-81d34a2ea607 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.541723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5878a-bbca-422a-b7fc-81d34a2ea607 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.542000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5878a-bbca-422a-b7fc-81d34a2ea607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.575162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6ad5878a-bbca-422a-b7fc-81d34a2ea607 None None] MFA Rules not processed for user `5e2a56d31d6b4519a45467748fc38c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:20.576486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6ad5878a-bbca-422a-b7fc-81d34a2ea607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:20.642873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 971/1922] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:20.647254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.709087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6502692-8899-485d-9faf-bde363561cb5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.713839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.714469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.714750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.715022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.715652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.717577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.828493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 952/1923] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:20.832608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5878a-bbca-422a-b7fc-81d34a2ea607 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.844135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5878a-bbca-422a-b7fc-81d34a2ea607 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.871296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:20.902055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.902723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.902998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.903268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.904077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.905972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.037390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 972/1924] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:21.041563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.052319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ceb05-8e7d-426f-b8e1-e8365b9bfb2a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.057763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.058783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.058783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.059066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.059705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.061310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.073713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 953/1925] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:21.077876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.089373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38810-3b47-4c26-bfbb-f8855f5fc3ba admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.094292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10caa22a-4c85-4d52-9754-8b7b7197717f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.095035 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa22a-4c85-4d52-9754-8b7b7197717f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.095571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa22a-4c85-4d52-9754-8b7b7197717f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.095936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caa22a-4c85-4d52-9754-8b7b7197717f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.096778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caa22a-4c85-4d52-9754-8b7b7197717f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.099000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10caa22a-4c85-4d52-9754-8b7b7197717f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.117149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 973/1926] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:21.118096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.129256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733fd4f-037e-46b2-8446-4148a40b911e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.133860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.134698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.134980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.135258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] PATH_INFO: `/v3/projects/c23d722872f947169fdd74abb45e2cbc/users/6d26536f9b914130be2670a9352845ac/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.143095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:21.150131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23d722872f947169fdd74abb45e2cbc, user_id=6d26536f9b914130be2670a9352845ac, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.152814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.236355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 954/1927] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:21 2022] PUT /identity/v3/projects/c23d722872f947169fdd74abb45e2cbc/users/6d26536f9b914130be2670a9352845ac/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:21.247024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057fb6a-3446-447a-94bd-e6dd5886525e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.247325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057fb6a-3446-447a-94bd-e6dd5886525e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.247602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057fb6a-3446-447a-94bd-e6dd5886525e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.276568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3057fb6a-3446-447a-94bd-e6dd5886525e None None] MFA Rules not processed for user `6d26536f9b914130be2670a9352845ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:21.277792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3057fb6a-3446-447a-94bd-e6dd5886525e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:21.344345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 974/1928] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:21.349149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.413491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b410737-c5fc-427e-8b3d-db26a7253462 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.420389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.420998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.421274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.421540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.422170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.424302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.495170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 955/1929] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:21 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:21.499090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057fb6a-3446-447a-94bd-e6dd5886525e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.510732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057fb6a-3446-447a-94bd-e6dd5886525e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.569838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.570463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.570739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.571008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.571879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.573826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.658452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 975/1930] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:21 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:21.662404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.674259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98027e6a-5d69-4ee5-ad39-99160d543bac admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.679115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.679781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.680065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.680340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.681292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.682944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.694989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 956/1931] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:21.699474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.710082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd37a0-2be0-47b6-8a3b-5a262e39b28c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.715889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.716598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.716877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.717147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] PATH_INFO: `/v3/projects/5b751cb9c8ca4ef0a36a73d07cc620e1/users/a167d0a28dff4445bbd6fbbf63deb694/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.733792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b751cb9c8ca4ef0a36a73d07cc620e1, user_id=a167d0a28dff4445bbd6fbbf63deb694, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.736915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.807781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 976/1932] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:21 2022] PUT /identity/v3/projects/5b751cb9c8ca4ef0a36a73d07cc620e1/users/a167d0a28dff4445bbd6fbbf63deb694/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:21.811546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.822079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177fb21-c031-4d28-8613-96ce6e53cc89 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.878568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b816d4-b08c-4f16-8741-14800b82eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.879204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b816d4-b08c-4f16-8741-14800b82eb26 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.879528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b816d4-b08c-4f16-8741-14800b82eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.879736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b816d4-b08c-4f16-8741-14800b82eb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.880466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b816d4-b08c-4f16-8741-14800b82eb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.881942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b816d4-b08c-4f16-8741-14800b82eb26 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.892987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 957/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:21.896811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:21.903078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:21.908536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64078c92-ab27-4bbf-8eb9-202818e0f48c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.913202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.913895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.919786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.919786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] PATH_INFO: `/v3/projects/5b751cb9c8ca4ef0a36a73d07cc620e1/users/a167d0a28dff4445bbd6fbbf63deb694/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.919786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b751cb9c8ca4ef0a36a73d07cc620e1, user_id=a167d0a28dff4445bbd6fbbf63deb694, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.919786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.930376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 977/1934] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:21 2022] PUT /identity/v3/projects/5b751cb9c8ca4ef0a36a73d07cc620e1/users/a167d0a28dff4445bbd6fbbf63deb694/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:21.940291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.940582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.940858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.974927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] MFA Rules not processed for user `a167d0a28dff4445bbd6fbbf63deb694`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:21.976158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:22.025886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:22.055840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 958/1935] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:22.067982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:22.073447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:22.148127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3de32-9425-48f4-922a-9c4f0d84763d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:22.149194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:22.150485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:22.150794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:22.151066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:22.161085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:22.162540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:22.173063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 978/1936] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:22.339148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:22.341881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:22.357286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f0d4-2097-45dd-bf6d-2d2551ee8718 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:22.361350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:22.361960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:22.363465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:22.363465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:22.424903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:22.426583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:22.436217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 959/1937] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:28.676666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:28.686204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddca549-56c8-492e-a56b-3756318c179e service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:28.697129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e86349-6a38-4584-8833-1fe4876508c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:28.697129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e86349-6a38-4584-8833-1fe4876508c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:28.697129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e86349-6a38-4584-8833-1fe4876508c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:28.697129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e86349-6a38-4584-8833-1fe4876508c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:28.697758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-57e86349-6a38-4584-8833-1fe4876508c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:28.765247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e86349-6a38-4584-8833-1fe4876508c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:28.766693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e86349-6a38-4584-8833-1fe4876508c4 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:28.776376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 979/1938] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:30.002998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.005469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:30.099365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acedc9ab-3555-4846-a36a-55d469124475 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.109152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.109152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.109152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.109152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] PATH_INFO: `/v3/users/2cbdccd20963495198088617073d7a58` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.109561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cbdccd20963495198088617073d7a58)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.109561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.322739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cbdccd20963495198088617073d7a58'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.396728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cbdccd20963495198088617073d7a58'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.419551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cbdccd20963495198088617073d7a58'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.431609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cbdccd20963495198088617073d7a58'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.431609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cbdccd20963495198088617073d7a58'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.434055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:30.439521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 960/1939] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:30 2022] DELETE /identity/v3/users/2cbdccd20963495198088617073d7a58 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:30.443781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.444295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.444386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.496625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:30.500212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:30.595577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 980/1940] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:30.607181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.608805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:30.668261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3a86-564c-45ff-8716-c24600c06d79 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.671823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.672432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.672713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.672987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.682579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.684004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.693446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 961/1941] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:30.970990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.972595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:30.986122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa6447-cf63-459b-bd69-51ae762367a2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.990765 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.991381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.991660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.991920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] PATH_INFO: `/v3/users/f558e73b196d46229f87873b8ecb73ac` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.993021 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f558e73b196d46229f87873b8ecb73ac)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.995113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.019054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:31.096056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.098537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.123508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49018a6f-f7aa-43c1-95a8-eeb3c907dc45 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.128501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.129274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.129615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.129891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] PATH_INFO: `/v3/projects/2c8d9d99397b46dab6430b8aa7bc1a33` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.131112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8d9d99397b46dab6430b8aa7bc1a33)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.133128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.166618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f558e73b196d46229f87873b8ecb73ac'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.213418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f558e73b196d46229f87873b8ecb73ac'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.235968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f558e73b196d46229f87873b8ecb73ac'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.236866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f558e73b196d46229f87873b8ecb73ac'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.237544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-71b7479c-5d66-4211-a086-08bb749a1054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f558e73b196d46229f87873b8ecb73ac'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.246058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8d9d99397b46dab6430b8aa7bc1a33', 'request_id': 'req-4d1198ba-57fe-4769-affd-2111aa6bb513'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.252425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 981/1942] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:30 2022] DELETE /identity/v3/users/f558e73b196d46229f87873b8ecb73ac => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:31.255220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8d9d99397b46dab6430b8aa7bc1a33', 'request_id': 'req-4d1198ba-57fe-4769-affd-2111aa6bb513'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.257449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.257735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.258046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.279269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:31.280476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.343183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 982/1943] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:31.353337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.354882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.372103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8d9d99397b46dab6430b8aa7bc1a33', 'request_id': 'req-4d1198ba-57fe-4769-affd-2111aa6bb513'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.386250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] The token cache is being invalidate because project 2c8d9d99397b46dab6430b8aa7bc1a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:31.386564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.387748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-4d1198ba-57fe-4769-affd-2111aa6bb513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:31.397470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 962/1944] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:31 2022] DELETE /identity/v3/projects/2c8d9d99397b46dab6430b8aa7bc1a33 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:31.403635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e74ff-213a-434c-b48c-ca1500bb9316 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.404116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e74ff-213a-434c-b48c-ca1500bb9316 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.404379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e74ff-213a-434c-b48c-ca1500bb9316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.418815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dfb57-bfbd-44c8-9331-5ed17c1e4170 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.422917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.423710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.423991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.424261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.426481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.427952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-872e74ff-213a-434c-b48c-ca1500bb9316 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:31.431736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-872e74ff-213a-434c-b48c-ca1500bb9316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.440672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.441176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 963/1945] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:31.442223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.449568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0c5ff-c57f-4f8d-a7f3-a9306e1f11e3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.449857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0c5ff-c57f-4f8d-a7f3-a9306e1f11e3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.450133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0c5ff-c57f-4f8d-a7f3-a9306e1f11e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.458900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 983/1946] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:31.471621 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-70c0c5ff-c57f-4f8d-a7f3-a9306e1f11e3 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:31.472872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-70c0c5ff-c57f-4f8d-a7f3-a9306e1f11e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.483315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 964/1947] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:31.488495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.502111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba934c9-6633-476a-b20f-c61213a9ce88 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.507185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.508602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.508602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.508602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.509800 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:31.510151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.511830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.526037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 984/1948] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:31.531014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0c5ff-c57f-4f8d-a7f3-a9306e1f11e3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.542904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0c5ff-c57f-4f8d-a7f3-a9306e1f11e3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.548185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.548794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.549070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.549552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.550140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.552116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.656306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 965/1949] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:31 2022] POST /identity/v3/projects => generated 399 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:31.660585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.671951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b474b5-23e2-4985-b22b-febbb0bdc4a2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.713917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:31.783887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.784773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.785208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.785642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.786491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.788586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.890190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 985/1950] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:31 2022] POST /identity/v3/users => generated 300 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:31.893338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.904314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcebc4-afab-4042-9b59-87030e01adb8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.908867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.909454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.909731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.910001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.910668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.912281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.922631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 966/1951] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:31.926580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.937571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d2f2-0ad2-4bb4-a6f9-7fccd7d65a77 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.942187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.942812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.943095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.943454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.944095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.945683 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.956588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 986/1952] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:31.960872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.970862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08de8ae-36ba-4082-9f0a-f2f905393dee admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.978674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31adec5-59d5-47d6-94a0-4c98cc6ea0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.979314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31adec5-59d5-47d6-94a0-4c98cc6ea0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.979599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31adec5-59d5-47d6-94a0-4c98cc6ea0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.979878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31adec5-59d5-47d6-94a0-4c98cc6ea0eb admin admin] PATH_INFO: `/v3/projects/53b110aaaf5640699fff5f5eac6cff0d/users/f4b6a0bc083c480c9a6d1e1300067ab6/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.995355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.997903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:32.001162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31adec5-59d5-47d6-94a0-4c98cc6ea0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b110aaaf5640699fff5f5eac6cff0d, user_id=f4b6a0bc083c480c9a6d1e1300067ab6, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:32.003850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31adec5-59d5-47d6-94a0-4c98cc6ea0eb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:32.016933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b449c3f-f2a5-48da-b7de-9d30084d6cce admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:32.021990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:32.022603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:32.022956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:32.029396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] PATH_INFO: `/v3/projects/982ec12120f942de98a05f4af3534b85` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:32.029396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=982ec12120f942de98a05f4af3534b85)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:32.029396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:32.065451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 967/1953] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:31 2022] PUT /identity/v3/projects/53b110aaaf5640699fff5f5eac6cff0d/users/f4b6a0bc083c480c9a6d1e1300067ab6/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:32.076632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:32.076940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:32.077237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:32.104536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] MFA Rules not processed for user `f4b6a0bc083c480c9a6d1e1300067ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:32.105760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:32.109698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:32.156947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '982ec12120f942de98a05f4af3534b85', 'request_id': 'req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.169550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 968/1954] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:32 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:32.181463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:32.182834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:32.236247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '982ec12120f942de98a05f4af3534b85', 'request_id': 'req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.237126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b17cde-5d71-405c-8151-5ef476a6058a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:32.244266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '982ec12120f942de98a05f4af3534b85', 'request_id': 'req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.255443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] The token cache is being invalidate because project 982ec12120f942de98a05f4af3534b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:32.255726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.256387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.258047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 987/1955] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:31 2022] DELETE /identity/v3/projects/982ec12120f942de98a05f4af3534b85 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:32.262987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:32.264377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:32.289293 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:32.290647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:32.291573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:32.292364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:32.301290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:32.330613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7a6bb-79f1-44fc-a27e-3e2d5524cb77 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:32.335586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:32.336238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:32.336496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:32.336761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] PATH_INFO: `/v3/users/9a7fb44cb777405dbf93dee163c7a621` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:32.338726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a7fb44cb777405dbf93dee163c7a621)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:32.340638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:32.353598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:32.355066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:32.364543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 969/1956] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:32 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:32.518794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:32.534850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a7fb44cb777405dbf93dee163c7a621'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.659825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a7fb44cb777405dbf93dee163c7a621'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.673587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a7fb44cb777405dbf93dee163c7a621'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.674343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a7fb44cb777405dbf93dee163c7a621'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.674991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a6dcfe4f-6a0d-4642-9940-2a7aa724085d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a7fb44cb777405dbf93dee163c7a621'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:32.687953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 988/1957] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:32 2022] DELETE /identity/v3/users/9a7fb44cb777405dbf93dee163c7a621 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:33.237539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:33.250526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff54f10-0b44-4508-8630-2fca23df85c9 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:33.268219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:33.316039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:33.316678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:33.316955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:33.317225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] PATH_INFO: `/v3/projects/4872eba106ec4396997fa938cd410a41` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:33.318336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4872eba106ec4396997fa938cd410a41)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:33.320280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:33.436972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4872eba106ec4396997fa938cd410a41', 'request_id': 'req-d1b974d4-e620-4f24-8587-061937626aca'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:33.449729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4872eba106ec4396997fa938cd410a41', 'request_id': 'req-d1b974d4-e620-4f24-8587-061937626aca'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:33.545378 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4872eba106ec4396997fa938cd410a41', 'request_id': 'req-d1b974d4-e620-4f24-8587-061937626aca'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:33.556966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] The token cache is being invalidate because project 4872eba106ec4396997fa938cd410a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:33.557202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:33.557823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-d1b974d4-e620-4f24-8587-061937626aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:33.559407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 970/1958] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:33 2022] DELETE /identity/v3/projects/4872eba106ec4396997fa938cd410a41 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:33.564377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365212c-3b9d-4939-8a14-bb8c575f1ae4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:33.564840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365212c-3b9d-4939-8a14-bb8c575f1ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:33.564930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365212c-3b9d-4939-8a14-bb8c575f1ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:33.584941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f365212c-3b9d-4939-8a14-bb8c575f1ae4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:33.586130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f365212c-3b9d-4939-8a14-bb8c575f1ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:33.667240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 989/1959] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:33.669066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c4fe8-8cc2-4257-941f-6f0e7b7b2f12 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:33.669353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c4fe8-8cc2-4257-941f-6f0e7b7b2f12 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:33.669626 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c4fe8-8cc2-4257-941f-6f0e7b7b2f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:33.691659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a48c4fe8-8cc2-4257-941f-6f0e7b7b2f12 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:33.692990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a48c4fe8-8cc2-4257-941f-6f0e7b7b2f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:33.701704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 971/1960] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:33.706130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365212c-3b9d-4939-8a14-bb8c575f1ae4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:33.718087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365212c-3b9d-4939-8a14-bb8c575f1ae4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:33.723065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:33.723719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:33.724006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:33.724278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:33.724937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:33.725239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:33.726964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:33.740051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 990/1961] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:33.744966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c4fe8-8cc2-4257-941f-6f0e7b7b2f12 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:33.755832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c4fe8-8cc2-4257-941f-6f0e7b7b2f12 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:33.761701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:33.762661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:33.763154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:33.763480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:33.764230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:33.766397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:33.894980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 972/1962] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:33 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:33.899083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:33.911243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27212f37-f421-4299-a4c1-7c60eee4d14b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:33.970993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:33.971586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:33.971862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:33.972128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:33.972747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:33.974679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:34.085445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 991/1963] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:33 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:34.090823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:34.101523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063d3d6-8d0b-41d5-b151-860103c583a6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:34.109435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:34.110038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:34.110313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:34.110612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:34.111321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:34.112919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:34.124435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 973/1964] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:34.128493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:34.140215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ace9e2-f14b-4d88-bc6b-dbade0e1a791 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:34.144852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0e57b-cf33-43a2-ad39-aabce799a942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:34.145444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e57b-cf33-43a2-ad39-aabce799a942 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:34.145718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e57b-cf33-43a2-ad39-aabce799a942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:34.145983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e57b-cf33-43a2-ad39-aabce799a942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:34.146652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0e57b-cf33-43a2-ad39-aabce799a942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:34.148222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0e57b-cf33-43a2-ad39-aabce799a942 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:34.163950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 992/1965] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:34.168271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:34.179193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe6ac-5798-48e1-953d-dc72d8658626 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:34.184529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:34.185192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:34.185471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:34.186254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] PATH_INFO: `/v3/projects/99edc55b227143f8a539d293d72bf110/users/ebb70db8cca44c678082741c62e3ef44/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:34.202763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99edc55b227143f8a539d293d72bf110, user_id=ebb70db8cca44c678082741c62e3ef44, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:34.205462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:34.283445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 974/1966] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:34 2022] PUT /identity/v3/projects/99edc55b227143f8a539d293d72bf110/users/ebb70db8cca44c678082741c62e3ef44/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:34.288523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349698f-ecf6-4504-8832-6579368659ee None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:34.288810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349698f-ecf6-4504-8832-6579368659ee None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:34.289082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349698f-ecf6-4504-8832-6579368659ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:34.308737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5349698f-ecf6-4504-8832-6579368659ee None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:34.309915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5349698f-ecf6-4504-8832-6579368659ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:34.319597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:34.373509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 993/1967] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:34.383244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:34.385531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:34.432466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:34.443769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b72409-eb4c-46de-adf6-d13837532688 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:34.448162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:34.448754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:34.449029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:34.449299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:34.458135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:34.460601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:34.470781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 975/1968] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:35.755581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39fb37-5d75-4d66-bb4c-15d4c7df6cc0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:35.756126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39fb37-5d75-4d66-bb4c-15d4c7df6cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:35.756126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39fb37-5d75-4d66-bb4c-15d4c7df6cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:35.785161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ca39fb37-5d75-4d66-bb4c-15d4c7df6cc0 None None] MFA Rules not processed for user `ebb70db8cca44c678082741c62e3ef44`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:35.786406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ca39fb37-5d75-4d66-bb4c-15d4c7df6cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:35.854370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 994/1969] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:35.858700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:35.868556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4ad6f-5714-4513-8986-4e99b44fef1d service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:35.874062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:35.874670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:35.874953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:35.875262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:35.875898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:35.877798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:35.969248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 976/1970] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:35 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:35.973299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39fb37-5d75-4d66-bb4c-15d4c7df6cc0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:35.984688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39fb37-5d75-4d66-bb4c-15d4c7df6cc0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:36.042144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:36.042773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:36.043050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:36.043318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:36.043930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:36.045821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:36.142804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 995/1971] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:35 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:36.144334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:36.156754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb949e-f1bf-4f5e-b0be-cbec5708034a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:36.161780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:36.162516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:36.162862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:36.163305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:36.164024 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:36.167196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:36.179865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 977/1972] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:36.184603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:36.200468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afca78-7118-40fe-bc7a-2c0770cf1593 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:36.205433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:36.206084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:36.206374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:36.206659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] PATH_INFO: `/v3/projects/e5650dee65484f59884ef3b1aa90e934/users/4e85bc60be8840f0ab33734649ba8019/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:36.225652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5650dee65484f59884ef3b1aa90e934, user_id=4e85bc60be8840f0ab33734649ba8019, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:36.229197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:36.328941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 996/1973] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:36 2022] PUT /identity/v3/projects/e5650dee65484f59884ef3b1aa90e934/users/4e85bc60be8840f0ab33734649ba8019/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:36.332768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:36.343572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa2f6e-19f3-4c36-a846-a442b1f7d3af admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:36.400812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4374544-99bb-4073-a962-5d9d44cd9110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:36.401419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4374544-99bb-4073-a962-5d9d44cd9110 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:36.401696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4374544-99bb-4073-a962-5d9d44cd9110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:36.402105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4374544-99bb-4073-a962-5d9d44cd9110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:36.402776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4374544-99bb-4073-a962-5d9d44cd9110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:36.404415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4374544-99bb-4073-a962-5d9d44cd9110 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:36.414118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 978/1974] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:36.419182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:36.430721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58efb-29f3-4e18-9811-2505e9282aa2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:36.435555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:36.436274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:36.436621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:36.436960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] PATH_INFO: `/v3/projects/e5650dee65484f59884ef3b1aa90e934/users/4e85bc60be8840f0ab33734649ba8019/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:36.439014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5650dee65484f59884ef3b1aa90e934, user_id=4e85bc60be8840f0ab33734649ba8019, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:36.441692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:36.448825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:36.454242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 997/1975] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:36 2022] PUT /identity/v3/projects/e5650dee65484f59884ef3b1aa90e934/users/4e85bc60be8840f0ab33734649ba8019/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:37.715932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:37.716492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:37.722045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:37.766115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] MFA Rules not processed for user `4e85bc60be8840f0ab33734649ba8019`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:37.767898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:37.886231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 979/1976] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:37.895162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:37.897009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:37.974936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82678a90-ff45-4d28-8fea-a7e015ed095a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:37.979061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed6c61-8976-428b-a6be-213c9232f711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:37.979264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed6c61-8976-428b-a6be-213c9232f711 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:37.979545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed6c61-8976-428b-a6be-213c9232f711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:37.979818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed6c61-8976-428b-a6be-213c9232f711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:38.037806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed6c61-8976-428b-a6be-213c9232f711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:38.039189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed6c61-8976-428b-a6be-213c9232f711 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:38.048649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 998/1977] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:38.956497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:38.958889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:39.028234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:39.068200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aac2d8-4f48-41cb-8bdb-820269c7a771 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:39.077753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:39.077753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:39.077753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:39.078115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] PATH_INFO: `/v3/users/5e2a56d31d6b4519a45467748fc38c8b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:39.079624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2a56d31d6b4519a45467748fc38c8b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:39.081717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:39.303799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2a56d31d6b4519a45467748fc38c8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:39.395791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2a56d31d6b4519a45467748fc38c8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:39.410259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2a56d31d6b4519a45467748fc38c8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:39.418079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2a56d31d6b4519a45467748fc38c8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:39.418754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2a56d31d6b4519a45467748fc38c8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:39.425100 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 980/1978] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:38 2022] DELETE /identity/v3/users/5e2a56d31d6b4519a45467748fc38c8b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:39.429756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:39.430627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:39.430627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:39.450620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:39.451827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:39.512541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 999/1979] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:39.522127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:39.523611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:39.582004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0285-f6f1-409c-b220-058bc9ef9eed admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:39.585850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:39.586466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:39.586745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:39.587006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:39.595847 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:39.597264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:39.607360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 981/1980] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:40.037204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:40.038681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:40.052210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5faec-9acd-4d55-a08e-c7d67e773996 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:40.056791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:40.057421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:40.057704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:40.057980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] PATH_INFO: `/v3/projects/65487b705788497385ca4ddae01cc5e1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:40.059838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65487b705788497385ca4ddae01cc5e1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:40.061743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:40.151644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:40.153738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65487b705788497385ca4ddae01cc5e1', 'request_id': 'req-a88821e9-5c78-44e0-9a3f-970608090a04'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.214918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65487b705788497385ca4ddae01cc5e1', 'request_id': 'req-a88821e9-5c78-44e0-9a3f-970608090a04'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.238936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65487b705788497385ca4ddae01cc5e1', 'request_id': 'req-a88821e9-5c78-44e0-9a3f-970608090a04'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.259278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] The token cache is being invalidate because project 65487b705788497385ca4ddae01cc5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:40.259580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.260390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.262899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1000/1981] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:40 2022] DELETE /identity/v3/projects/65487b705788497385ca4ddae01cc5e1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:40.266715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:40.277467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2b732-d383-4d50-b0fb-d4dd54d9cdd5 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:40.336281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:40.344346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:40.344979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:40.345261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:40.345537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] PATH_INFO: `/v3/users/6d26536f9b914130be2670a9352845ac` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:40.346703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d26536f9b914130be2670a9352845ac)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:40.348643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:40.507602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d26536f9b914130be2670a9352845ac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.578411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d26536f9b914130be2670a9352845ac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.592325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d26536f9b914130be2670a9352845ac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.600263 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d26536f9b914130be2670a9352845ac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.601010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d26536f9b914130be2670a9352845ac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:40.610781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 982/1982] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:40 2022] DELETE /identity/v3/users/6d26536f9b914130be2670a9352845ac => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:41.175419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:41.177464 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:41.198017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88821e9-5c78-44e0-9a3f-970608090a04 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:41.261131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:41.263795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:41.264094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:41.264372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] PATH_INFO: `/v3/projects/c23d722872f947169fdd74abb45e2cbc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:41.265654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23d722872f947169fdd74abb45e2cbc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:41.267604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:41.345975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23d722872f947169fdd74abb45e2cbc', 'request_id': 'req-13d8dff9-6874-4101-bb3f-e54fe2b183a1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.389423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23d722872f947169fdd74abb45e2cbc', 'request_id': 'req-13d8dff9-6874-4101-bb3f-e54fe2b183a1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.397089 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23d722872f947169fdd74abb45e2cbc', 'request_id': 'req-13d8dff9-6874-4101-bb3f-e54fe2b183a1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.411109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] The token cache is being invalidate because project c23d722872f947169fdd74abb45e2cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:41.411465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.412133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.413797 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1001/1983] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:41 2022] DELETE /identity/v3/projects/c23d722872f947169fdd74abb45e2cbc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:41.418446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:41.420412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:41.438385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe70625-ca2a-497f-870e-e1b99b0a655a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:41.512174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:41.512816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:41.522144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:41.522144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] PATH_INFO: `/v3/users/a167d0a28dff4445bbd6fbbf63deb694` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:41.522144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a167d0a28dff4445bbd6fbbf63deb694)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:41.522144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:41.718922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a167d0a28dff4445bbd6fbbf63deb694'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.830254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a167d0a28dff4445bbd6fbbf63deb694'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.843716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a167d0a28dff4445bbd6fbbf63deb694'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.848090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:41.851042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a167d0a28dff4445bbd6fbbf63deb694'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.851695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5b25c89d-271f-4b29-8618-1acb4c05b1d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a167d0a28dff4445bbd6fbbf63deb694'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:41.858370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 983/1984] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:41 2022] DELETE /identity/v3/users/a167d0a28dff4445bbd6fbbf63deb694 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:42.271380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:42.272782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:42.296840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8dff9-6874-4101-bb3f-e54fe2b183a1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:42.362522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:42.363154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:42.363431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:42.363702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] PATH_INFO: `/v3/projects/5b751cb9c8ca4ef0a36a73d07cc620e1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:42.364830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b751cb9c8ca4ef0a36a73d07cc620e1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:42.366905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:42.375635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:42.475048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b751cb9c8ca4ef0a36a73d07cc620e1', 'request_id': 'req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:42.543565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b751cb9c8ca4ef0a36a73d07cc620e1', 'request_id': 'req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:42.556071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b751cb9c8ca4ef0a36a73d07cc620e1', 'request_id': 'req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:42.567671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] The token cache is being invalidate because project 5b751cb9c8ca4ef0a36a73d07cc620e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:42.570203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:42.570203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6e51aed4-f22a-435d-ace9-c68a9c1aa9d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:42.570203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1002/1985] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:42 2022] DELETE /identity/v3/projects/5b751cb9c8ca4ef0a36a73d07cc620e1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:42.576648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c416-2ad4-45b0-b8b9-3bebad547070 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:42.577122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c416-2ad4-45b0-b8b9-3bebad547070 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:42.577353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c416-2ad4-45b0-b8b9-3bebad547070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:42.597232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7553c416-2ad4-45b0-b8b9-3bebad547070 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:42.598572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7553c416-2ad4-45b0-b8b9-3bebad547070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:42.672587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 984/1986] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:42.682263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428929-80d3-4a81-80bb-c59f4dcc445f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:42.682581 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428929-80d3-4a81-80bb-c59f4dcc445f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:42.683389 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428929-80d3-4a81-80bb-c59f4dcc445f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:42.719027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d8428929-80d3-4a81-80bb-c59f4dcc445f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:42.720263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d8428929-80d3-4a81-80bb-c59f4dcc445f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:42.729157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1003/1987] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:42.734200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c416-2ad4-45b0-b8b9-3bebad547070 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:42.744814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c416-2ad4-45b0-b8b9-3bebad547070 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:42.750499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:42.751111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:42.751386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:42.751654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:42.752294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:42.752590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:42.754188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:42.765885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 985/1988] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:42.770471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8428929-80d3-4a81-80bb-c59f4dcc445f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:42.779751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8428929-80d3-4a81-80bb-c59f4dcc445f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:42.784363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:42.784947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:42.785223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:42.786978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:42.786978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:42.788008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:42.881862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1004/1989] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:42 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:42.886056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:42.895970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42379-748e-4370-8728-9e41d0358810 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:42.956402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:42.957038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:42.957317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:42.957589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:42.958523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:42.960424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:43.061169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 986/1990] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:42 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:43.064891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:43.076309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ac1e1-5684-4099-b6be-cf021b1462be admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:43.101760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:43.102611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:43.102993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:43.103367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:43.104127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:43.106146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:43.117216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1005/1991] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:43.121790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:43.135684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe0657-b286-49e7-abbc-b48f38517732 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:43.141604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586dc35-d665-4534-a30d-8652fc491542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:43.142345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586dc35-d665-4534-a30d-8652fc491542 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:43.142762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586dc35-d665-4534-a30d-8652fc491542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:43.143196 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586dc35-d665-4534-a30d-8652fc491542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:43.143927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586dc35-d665-4534-a30d-8652fc491542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:43.145936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586dc35-d665-4534-a30d-8652fc491542 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:43.159181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 987/1992] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:43.163311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:43.174328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e567cb-62dc-4256-9bb9-0a501e31e20c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:43.179203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:43.179881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:43.180159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:43.180428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] PATH_INFO: `/v3/projects/bbd654c229774aa8a91164c0b097bd26/users/aee833b6e2444bf5a298ad45d2afeea8/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:43.196011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd654c229774aa8a91164c0b097bd26, user_id=aee833b6e2444bf5a298ad45d2afeea8, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:43.199339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:43.283281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1006/1993] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:43 2022] PUT /identity/v3/projects/bbd654c229774aa8a91164c0b097bd26/users/aee833b6e2444bf5a298ad45d2afeea8/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:43.301008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:43.301289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:43.301556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:43.331166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] MFA Rules not processed for user `aee833b6e2444bf5a298ad45d2afeea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:43.332374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:43.382160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:43.399964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 988/1994] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:43.413110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:43.414824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:43.506479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d7e39-0466-416c-97f0-236cad91f127 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:43.510321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:43.511497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:43.511962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:43.512270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:43.521818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:43.523261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:43.538949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1007/1995] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:53.447232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:53.531615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:57.859767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:57.861217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:57.925294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb10d9-c922-4d30-90cc-26cbf6b5e5cf None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:57.929490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9df413-f150-41bf-b19b-16f2136dec7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:57.930158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9df413-f150-41bf-b19b-16f2136dec7d service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:57.930473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9df413-f150-41bf-b19b-16f2136dec7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:57.930749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9df413-f150-41bf-b19b-16f2136dec7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:57.933285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0a9df413-f150-41bf-b19b-16f2136dec7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:58.017600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9df413-f150-41bf-b19b-16f2136dec7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:58.019080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9df413-f150-41bf-b19b-16f2136dec7d service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:58.038558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 989/1996] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:58.383409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:58.384956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:58.457126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21664d4-a154-4c35-93df-684dc32bfaa8 service glance] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:58.462163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:58.463015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.463428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.463593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] PATH_INFO: `/v3/users/f4b6a0bc083c480c9a6d1e1300067ab6` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.464729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b6a0bc083c480c9a6d1e1300067ab6)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:58.466859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:58.685027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b6a0bc083c480c9a6d1e1300067ab6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:58.794466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b6a0bc083c480c9a6d1e1300067ab6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:58.816261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b6a0bc083c480c9a6d1e1300067ab6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:58.817140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b6a0bc083c480c9a6d1e1300067ab6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:58.817811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b6a0bc083c480c9a6d1e1300067ab6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:58.839282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1008/1997] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:58 2022] DELETE /identity/v3/users/f4b6a0bc083c480c9a6d1e1300067ab6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:58.841185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.841552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.841892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.866212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:58.868044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:58.944679 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 990/1998] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:58.958394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:58.961935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:59.031725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff90bc-f101-488e-97c1-ac604d12743e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:59.037136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd85e4d-0029-4cac-9c4c-cf81dea48333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:59.037815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd85e4d-0029-4cac-9c4c-cf81dea48333 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:59.038322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd85e4d-0029-4cac-9c4c-cf81dea48333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:59.043487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd85e4d-0029-4cac-9c4c-cf81dea48333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:59.049467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd85e4d-0029-4cac-9c4c-cf81dea48333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:59.050928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd85e4d-0029-4cac-9c4c-cf81dea48333 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:59.060901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1009/1999] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:59.689016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:59.690776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:59.708437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b8cc0-1302-4bc4-b809-bb87a5978798 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:59.711859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:59.712492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:59.712771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:59.713043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] PATH_INFO: `/v3/projects/53b110aaaf5640699fff5f5eac6cff0d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:59.714806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b110aaaf5640699fff5f5eac6cff0d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:59.716821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:59.813884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:59.817119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b110aaaf5640699fff5f5eac6cff0d', 'request_id': 'req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:59.824932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b110aaaf5640699fff5f5eac6cff0d', 'request_id': 'req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:59.886136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b110aaaf5640699fff5f5eac6cff0d', 'request_id': 'req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:59.922004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] The token cache is being invalidate because project 53b110aaaf5640699fff5f5eac6cff0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:59.922403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:59.923197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3e9c67b6-7aa4-4fd6-ae2c-56af720d1ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:59.925250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 991/2000] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:59 2022] DELETE /identity/v3/projects/53b110aaaf5640699fff5f5eac6cff0d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:59.931456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ee88-75ac-4d33-b68c-f5b313eed358 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:59.932390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ee88-75ac-4d33-b68c-f5b313eed358 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:59.932938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ee88-75ac-4d33-b68c-f5b313eed358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:59.956594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4249ee88-75ac-4d33-b68c-f5b313eed358 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:59.958127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4249ee88-75ac-4d33-b68c-f5b313eed358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:59.977507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:00.034387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1010/2001] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:00.039166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f317b-a6aa-4d10-8a10-7771100753ab None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.039455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f317b-a6aa-4d10-8a10-7771100753ab None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.039726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f317b-a6aa-4d10-8a10-7771100753ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.060570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c38f317b-a6aa-4d10-8a10-7771100753ab None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:00.067117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c38f317b-a6aa-4d10-8a10-7771100753ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:00.070860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 992/2002] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:00.075925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249ee88-75ac-4d33-b68c-f5b313eed358 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:00.086406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249ee88-75ac-4d33-b68c-f5b313eed358 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:00.093362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:00.094069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.094470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.094747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.095397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:00.095696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:00.097363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:00.115382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f317b-a6aa-4d10-8a10-7771100753ab None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:00.116083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1011/2003] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:00.133065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f317b-a6aa-4d10-8a10-7771100753ab None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:00.140463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:00.141871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.142442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.142890 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.143706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:00.146380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:00.274254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 993/2004] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:00 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:00.274406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:00.298452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757f2d2-601a-4fa5-846e-a3e223a8a2bf admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:00.403887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:00.405206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.406055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.406574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.407439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:00.410703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:00.557192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1012/2005] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:00 2022] POST /identity/v3/users => generated 292 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:00.561300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:00.572281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c866c-90a8-473a-ac93-63cb060fb278 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:00.579806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:00.579806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.579806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.579806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.580163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:00.580869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:00.592066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 994/2006] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:00.596246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:00.606870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98a6c9-bdfc-4bfe-8d66-267163020e4e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:00.611614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbae2a-f759-42b7-829d-0cbd639bf186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:00.612205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbae2a-f759-42b7-829d-0cbd639bf186 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.612486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbae2a-f759-42b7-829d-0cbd639bf186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.612760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbae2a-f759-42b7-829d-0cbd639bf186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.613400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dbae2a-f759-42b7-829d-0cbd639bf186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:00.615022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dbae2a-f759-42b7-829d-0cbd639bf186 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:00.625775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1013/2007] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:00.629828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:00.640041 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e4f7c-dacf-4f0b-92e1-f637cbdaf96a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:00.645692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:00.646384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.646655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.646932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] PATH_INFO: `/v3/projects/31bdca178d874d499b1492305ad45a9c/users/747b60cf32a14b93bee399fce7a9b54b/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.663900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31bdca178d874d499b1492305ad45a9c, user_id=747b60cf32a14b93bee399fce7a9b54b, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:00.667125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:00.733625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 995/2008] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:00 2022] PUT /identity/v3/projects/31bdca178d874d499b1492305ad45a9c/users/747b60cf32a14b93bee399fce7a9b54b/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:00.738748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46621d2-dc7f-4f4d-9eb0-e24335421725 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.739042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46621d2-dc7f-4f4d-9eb0-e24335421725 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.739322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46621d2-dc7f-4f4d-9eb0-e24335421725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.758764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a46621d2-dc7f-4f4d-9eb0-e24335421725 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:00.759906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a46621d2-dc7f-4f4d-9eb0-e24335421725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:00.822060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1014/2009] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:00.836280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:00.838030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:00.896219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f13a-bb4e-4fdf-bbc7-454febd4bb92 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:00.899993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:00.900595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:00.901427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:00.901699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:00.910903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:00.912326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:00.923668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 996/2010] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:06.415488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ab56-eadf-4282-80d7-28d3fcb56b78 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:06.416033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ab56-eadf-4282-80d7-28d3fcb56b78 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:06.416033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ab56-eadf-4282-80d7-28d3fcb56b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:06.456587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2651ab56-eadf-4282-80d7-28d3fcb56b78 None None] MFA Rules not processed for user `747b60cf32a14b93bee399fce7a9b54b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:06.459272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2651ab56-eadf-4282-80d7-28d3fcb56b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:06.485808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:06.542191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1015/2011] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:06.544502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:06.565887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc6f7-a75a-48bb-9c34-2b2f0f238f7a service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:06.572515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:06.573306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:06.583793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:06.583793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:06.583793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:06.585593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:06.696781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 997/2012] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:06 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:06.701066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ab56-eadf-4282-80d7-28d3fcb56b78 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:06.719476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ab56-eadf-4282-80d7-28d3fcb56b78 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:06.836379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:06.837471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:06.837756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:06.839070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:06.839784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:06.841934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:06.932740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1016/2013] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:06 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:06.933970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:06.961366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b482f0-9201-4475-928f-3d281b542ead admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:06.976143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:06.976991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:06.977424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:06.977876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:06.978908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:06.982523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:06.999999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 998/2014] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:07.004189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:07.018057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f5ea1-2051-49e8-955b-7adce2bce0c2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:07.023824 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:07.024504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:07.024785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:07.025060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] PATH_INFO: `/v3/projects/d6ff6b4542fb4445865c6a9cd7923b73/users/44630f5248e04ed99c59b42397a6c273/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:07.057650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff6b4542fb4445865c6a9cd7923b73, user_id=44630f5248e04ed99c59b42397a6c273, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:07.060415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:07.137678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1017/2015] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:07 2022] PUT /identity/v3/projects/d6ff6b4542fb4445865c6a9cd7923b73/users/44630f5248e04ed99c59b42397a6c273/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:07.141683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:07.151916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3368394-9832-4fd8-aa22-4b7aee2a37a6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:07.194906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:07.221096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:07.221096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:07.221096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:07.221096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:07.221502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:07.223281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:07.238425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 999/2016] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:07.243908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:07.255798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2accc-8184-4848-b2ca-f54f145f592d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:07.261737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118af7d5-bfc1-4b15-b97e-cefc739ac985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:07.262578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118af7d5-bfc1-4b15-b97e-cefc739ac985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:07.262937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118af7d5-bfc1-4b15-b97e-cefc739ac985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:07.263261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118af7d5-bfc1-4b15-b97e-cefc739ac985 admin admin] PATH_INFO: `/v3/projects/d6ff6b4542fb4445865c6a9cd7923b73/users/44630f5248e04ed99c59b42397a6c273/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:07.265315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118af7d5-bfc1-4b15-b97e-cefc739ac985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff6b4542fb4445865c6a9cd7923b73, user_id=44630f5248e04ed99c59b42397a6c273, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:07.268148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118af7d5-bfc1-4b15-b97e-cefc739ac985 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:07.279403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1018/2017] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:07 2022] PUT /identity/v3/projects/d6ff6b4542fb4445865c6a9cd7923b73/users/44630f5248e04ed99c59b42397a6c273/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:11.170769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:11.172561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:11.188930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545c793-4930-429a-95d3-af5c0ebadae8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:11.261021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:11.261728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:11.262012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:11.262419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] PATH_INFO: `/v3/users/5f59c6c53a67481e83d673251d469e13` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:11.263746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f59c6c53a67481e83d673251d469e13)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:11.265915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:11.462786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f59c6c53a67481e83d673251d469e13'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:11.555437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f59c6c53a67481e83d673251d469e13'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:11.604589 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f59c6c53a67481e83d673251d469e13'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:11.630780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f59c6c53a67481e83d673251d469e13'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:11.633029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f59c6c53a67481e83d673251d469e13'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:11.651416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1000/2018] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:11 2022] DELETE /identity/v3/users/5f59c6c53a67481e83d673251d469e13 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:11.655720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:11.656667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:11.657630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:11.703198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:11.705842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:11.841802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1019/2019] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:11.848429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:11.849900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:11.982588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b36a5-6114-411c-93ae-d902b6173dd2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:11.986913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:11.987966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:11.988248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:11.988517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:12.003747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:12.005371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:12.011774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:12.019094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1001/2020] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:12.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:12.596291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:12.626076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b0e1-2e17-4a5c-a96f-bbd1307c5781 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:12.637998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:12.643602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:12.643602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:12.643602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] PATH_INFO: `/v3/projects/f23d6ea67ba84884b28ecad806b75589` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:12.643602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f23d6ea67ba84884b28ecad806b75589)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:12.646114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:12.654208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:12.786798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f23d6ea67ba84884b28ecad806b75589', 'request_id': 'req-6b90b45a-1763-4f86-bad1-a77ef4686eb3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:12.876208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f23d6ea67ba84884b28ecad806b75589', 'request_id': 'req-6b90b45a-1763-4f86-bad1-a77ef4686eb3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:12.887108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f23d6ea67ba84884b28ecad806b75589', 'request_id': 'req-6b90b45a-1763-4f86-bad1-a77ef4686eb3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:12.919726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] The token cache is being invalidate because project f23d6ea67ba84884b28ecad806b75589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:12.920019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:12.921371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:12.929129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1020/2021] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:12 2022] DELETE /identity/v3/projects/f23d6ea67ba84884b28ecad806b75589 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:12.933216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:12.945348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7d58f-ebcc-4f79-b26c-b182e719e5be service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:13.075690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:13.076612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:13.077062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:13.077534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] PATH_INFO: `/v3/users/cd00f6183cef43819c90d7fbd398da80` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:13.079271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd00f6183cef43819c90d7fbd398da80)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:13.083447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:13.326687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd00f6183cef43819c90d7fbd398da80'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:13.418260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd00f6183cef43819c90d7fbd398da80'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:13.441970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd00f6183cef43819c90d7fbd398da80'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:13.449873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd00f6183cef43819c90d7fbd398da80'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:13.450671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a55d2ad3-83f7-42fa-ad5a-3de1a46b85e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd00f6183cef43819c90d7fbd398da80'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:13.457322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1002/2022] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:12 2022] DELETE /identity/v3/users/cd00f6183cef43819c90d7fbd398da80 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:13.988663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:13.990682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.020883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90b45a-1763-4f86-bad1-a77ef4686eb3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.060687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.061159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.061244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.108576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] MFA Rules not processed for user `44630f5248e04ed99c59b42397a6c273`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:14.114298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.136012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.136662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.136977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.138860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] PATH_INFO: `/v3/projects/eb8b786327504e7da407c1137bd1e5d7` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.140952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8b786327504e7da407c1137bd1e5d7)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:14.142933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:14.217436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1003/2023] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:14.230203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.231691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.239250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:14.274979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8b786327504e7da407c1137bd1e5d7', 'request_id': 'req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:14.309687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d0fdf-1ea9-4752-8fc0-666117330821 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.355281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8b786327504e7da407c1137bd1e5d7', 'request_id': 'req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:14.362006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3078d97-35c5-46c4-a80b-233edf6ec3cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.362158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8b786327504e7da407c1137bd1e5d7', 'request_id': 'req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:14.362645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3078d97-35c5-46c4-a80b-233edf6ec3cb service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.362921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3078d97-35c5-46c4-a80b-233edf6ec3cb service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.363300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3078d97-35c5-46c4-a80b-233edf6ec3cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.365062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a3078d97-35c5-46c4-a80b-233edf6ec3cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.375981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] The token cache is being invalidate because project eb8b786327504e7da407c1137bd1e5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:14.376318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:14.377092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-59f3ee9b-70e9-4c36-b4fb-e2090eb00cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:14.378798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1021/2024] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:13 2022] DELETE /identity/v3/projects/eb8b786327504e7da407c1137bd1e5d7 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:14.383535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142f59-18b0-4d8d-9a2c-7b8b9808c2ef None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.383917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142f59-18b0-4d8d-9a2c-7b8b9808c2ef None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.384203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142f59-18b0-4d8d-9a2c-7b8b9808c2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.407491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-93142f59-18b0-4d8d-9a2c-7b8b9808c2ef None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:14.408720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-93142f59-18b0-4d8d-9a2c-7b8b9808c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.433452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3078d97-35c5-46c4-a80b-233edf6ec3cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:14.434967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3078d97-35c5-46c4-a80b-233edf6ec3cb service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:14.436468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:14.445061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1004/2025] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:14.476864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1022/2026] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:14.485318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f876f-b8a0-444f-9c97-ad8c5ee88191 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.485714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f876f-b8a0-444f-9c97-ad8c5ee88191 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.485991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f876f-b8a0-444f-9c97-ad8c5ee88191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.506575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-357f876f-b8a0-444f-9c97-ad8c5ee88191 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:14.507914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-357f876f-b8a0-444f-9c97-ad8c5ee88191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.516874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1005/2027] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:14.521230 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93142f59-18b0-4d8d-9a2c-7b8b9808c2ef None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.531578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93142f59-18b0-4d8d-9a2c-7b8b9808c2ef None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.536186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.536817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.537093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.537361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.537996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:14.538293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:14.539883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:14.552493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1023/2028] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:14.557552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f876f-b8a0-444f-9c97-ad8c5ee88191 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.566757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f876f-b8a0-444f-9c97-ad8c5ee88191 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.571469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75790892-7eb9-4a75-ad55-4ef4793488c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.572142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790892-7eb9-4a75-ad55-4ef4793488c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.576710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790892-7eb9-4a75-ad55-4ef4793488c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.576710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790892-7eb9-4a75-ad55-4ef4793488c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.576710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75790892-7eb9-4a75-ad55-4ef4793488c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:14.576710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75790892-7eb9-4a75-ad55-4ef4793488c2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:14.683335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1006/2029] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:14 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:14.687186 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.697399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687e851-ec3f-4dab-be55-dd898d11a950 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.731558 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeab8be-545a-4717-b831-abf967e0c9e8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.732030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeab8be-545a-4717-b831-abf967e0c9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.732511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeab8be-545a-4717-b831-abf967e0c9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.752226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4eeab8be-545a-4717-b831-abf967e0c9e8 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:14.753457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4eeab8be-545a-4717-b831-abf967e0c9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.768605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.769521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.770015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.770451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.771222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:14.773889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:14.812910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:14.836344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1007/2030] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:34:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:14.869699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1024/2031] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:14 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:14.871149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeab8be-545a-4717-b831-abf967e0c9e8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.888935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeab8be-545a-4717-b831-abf967e0c9e8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.896020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.898292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.898552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.903350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.903350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.903350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:14.903695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:14.903810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:14.927622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1008/2032] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:14.931791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.942814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a41d2d-fae0-48ef-886a-2c60f50fbf7e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.953077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.953976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.954800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.955219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:14.956618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:14.958649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:14.973264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1009/2033] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:14.974317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:14.986507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7390a-8e12-4fc4-905f-face700c4954 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.992544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:14.993288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:14.993658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:14.993958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e11c0-3663-4513-a3c0-f99e1103056d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:14.994580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] PATH_INFO: `/v3/projects/e9c6eacc575f4c2f8be3f14db5416cbd/users/b1b0f1bae473465bacf095aee6bcd2e0/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:15.004959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d7d1d-9e35-4ca0-afcd-b38b1c95c927 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:15.005575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d7d1d-9e35-4ca0-afcd-b38b1c95c927 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:15.005852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d7d1d-9e35-4ca0-afcd-b38b1c95c927 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:15.006124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d7d1d-9e35-4ca0-afcd-b38b1c95c927 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:15.016931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d7d1d-9e35-4ca0-afcd-b38b1c95c927 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:15.018444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d7d1d-9e35-4ca0-afcd-b38b1c95c927 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:15.028083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c6eacc575f4c2f8be3f14db5416cbd, user_id=b1b0f1bae473465bacf095aee6bcd2e0, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:15.028986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1025/2034] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:15.034184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:15.148699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1010/2035] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:14 2022] PUT /identity/v3/projects/e9c6eacc575f4c2f8be3f14db5416cbd/users/b1b0f1bae473465bacf095aee6bcd2e0/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:15.152324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da350143-4b8c-4b76-8d17-92d7dd5293d5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:15.152618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da350143-4b8c-4b76-8d17-92d7dd5293d5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:15.152890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da350143-4b8c-4b76-8d17-92d7dd5293d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:15.207661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-da350143-4b8c-4b76-8d17-92d7dd5293d5 None None] MFA Rules not processed for user `b1b0f1bae473465bacf095aee6bcd2e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:15.209242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-da350143-4b8c-4b76-8d17-92d7dd5293d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:15.239626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:15.306220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1026/2036] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:15.310392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:15.381585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3361c-32b5-460d-ae2b-f9f1fdcbdeea admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:15.387343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:15.387963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:15.388241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:15.388514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:15.389340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:15.391449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:15.511758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1011/2037] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:15 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:15.515524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da350143-4b8c-4b76-8d17-92d7dd5293d5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:15.529164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da350143-4b8c-4b76-8d17-92d7dd5293d5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:15.601148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:15.601754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:15.602029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:15.602296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:15.602986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:15.605635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:15.862142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1027/2038] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:15 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:15.863174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:15.877335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3f10-42a8-40f4-8e01-4941667a6776 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:15.883824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:15.899323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:15.899600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:15.900688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:15.901399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:15.903077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:15.924101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1012/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:15.930054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:15.950489 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3088-d5c6-4245-b9d7-397d7993d080 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:15.960146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:15.960823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:15.961163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:15.961441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] PATH_INFO: `/v3/projects/c04cd6b58f8a4348911e8c14f9fae540/users/94ae6e36b12241f3a211596212a2f4fd/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:15.984283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04cd6b58f8a4348911e8c14f9fae540, user_id=94ae6e36b12241f3a211596212a2f4fd, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:15.989356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:16.272712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1028/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:15 2022] PUT /identity/v3/projects/c04cd6b58f8a4348911e8c14f9fae540/users/94ae6e36b12241f3a211596212a2f4fd/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:16.277396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:16.292444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ad740-f3f4-4e4e-b9a2-8af99cbf1819 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:16.377132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81386042-9f87-4d1d-ab17-f289dbb1fbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:16.377745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81386042-9f87-4d1d-ab17-f289dbb1fbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:16.378072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81386042-9f87-4d1d-ab17-f289dbb1fbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:16.378370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81386042-9f87-4d1d-ab17-f289dbb1fbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:16.379018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81386042-9f87-4d1d-ab17-f289dbb1fbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:16.380602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81386042-9f87-4d1d-ab17-f289dbb1fbb7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:16.397945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1013/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:16.399307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:16.411998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf6d3b-5382-44d9-b220-66a749b1e502 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:16.420758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:16.421749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:16.422126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:16.422497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] PATH_INFO: `/v3/projects/c04cd6b58f8a4348911e8c14f9fae540/users/94ae6e36b12241f3a211596212a2f4fd/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:16.424656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04cd6b58f8a4348911e8c14f9fae540, user_id=94ae6e36b12241f3a211596212a2f4fd, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:16.427994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:16.450391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1029/2042] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:16 2022] PUT /identity/v3/projects/c04cd6b58f8a4348911e8c14f9fae540/users/94ae6e36b12241f3a211596212a2f4fd/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:16.455788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367bd0-0a38-4762-a574-4738c16b321f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:16.456075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367bd0-0a38-4762-a574-4738c16b321f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:16.457526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367bd0-0a38-4762-a574-4738c16b321f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:16.495831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:16.502442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-25367bd0-0a38-4762-a574-4738c16b321f None None] MFA Rules not processed for user `94ae6e36b12241f3a211596212a2f4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:16.503670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-25367bd0-0a38-4762-a574-4738c16b321f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:16.602820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1014/2043] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:16.615189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:16.617537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:16.718880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b515b-abb0-4494-8d0d-1badc87706dd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:16.723203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631c199-bd8d-49fb-ab56-b8d516dcab0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:16.723906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c199-bd8d-49fb-ab56-b8d516dcab0f service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:16.724194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c199-bd8d-49fb-ab56-b8d516dcab0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:16.724598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c199-bd8d-49fb-ab56-b8d516dcab0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:16.738510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631c199-bd8d-49fb-ab56-b8d516dcab0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:16.739979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631c199-bd8d-49fb-ab56-b8d516dcab0f service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:16.770130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1030/2044] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:18.256522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25367bd0-0a38-4762-a574-4738c16b321f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:18.338170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25367bd0-0a38-4762-a574-4738c16b321f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:18.349985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:18.351300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:18.351300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:18.351757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] PATH_INFO: `/v3/users/b1b0f1bae473465bacf095aee6bcd2e0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:18.353651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b0f1bae473465bacf095aee6bcd2e0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:18.359345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:18.746808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b0f1bae473465bacf095aee6bcd2e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:18.840432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b0f1bae473465bacf095aee6bcd2e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:18.859108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b0f1bae473465bacf095aee6bcd2e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:18.879430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b0f1bae473465bacf095aee6bcd2e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:18.880521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b0f1bae473465bacf095aee6bcd2e0'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:18.891680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1015/2045] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:18 2022] DELETE /identity/v3/users/b1b0f1bae473465bacf095aee6bcd2e0 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:18.896194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:18.902419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:18.902419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:18.936746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:18.939551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:18.958756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:19.045166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1031/2046] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:19.055024 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:19.057751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:19.120753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:19.220147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2bbe4-6eb7-4ed9-8ab3-0cb88eacc1b1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:19.225994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:19.226836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:19.227622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:19.227999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:19.271510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:19.273051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:19.291981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1016/2047] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:19.902448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:19.904811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:19.919304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5019371-14c4-4d9c-bd2e-29559aa6e2fe None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:19.929533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:19.930169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:19.930477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:19.931474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] PATH_INFO: `/v3/projects/e9c6eacc575f4c2f8be3f14db5416cbd` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:19.941352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c6eacc575f4c2f8be3f14db5416cbd)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:19.944280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:20.077725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c6eacc575f4c2f8be3f14db5416cbd', 'request_id': 'req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.170720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c6eacc575f4c2f8be3f14db5416cbd', 'request_id': 'req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.177804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c6eacc575f4c2f8be3f14db5416cbd', 'request_id': 'req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.210137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] The token cache is being invalidate because project e9c6eacc575f4c2f8be3f14db5416cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:20.210462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.211223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.214054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1032/2048] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:19 2022] DELETE /identity/v3/projects/e9c6eacc575f4c2f8be3f14db5416cbd => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:20.222959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:20.249735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad4821-4ba4-4edd-b948-7b0255dc418b service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:20.382317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:20.383146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:20.383666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:20.384658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] PATH_INFO: `/v3/users/94ae6e36b12241f3a211596212a2f4fd` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:20.386909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ae6e36b12241f3a211596212a2f4fd)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:20.389353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:20.674748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ae6e36b12241f3a211596212a2f4fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.760893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ae6e36b12241f3a211596212a2f4fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.788616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ae6e36b12241f3a211596212a2f4fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.802019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ae6e36b12241f3a211596212a2f4fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.803509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0341c228-6f09-47ae-87ab-d060d5197582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ae6e36b12241f3a211596212a2f4fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:20.814524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1017/2049] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:20 2022] DELETE /identity/v3/users/94ae6e36b12241f3a211596212a2f4fd => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:21.357511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:21.360501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:21.378024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25bd59-4bac-4c11-a19f-e0eb9c96d792 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:21.494419 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:21.495328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:21.495605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:21.495877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] PATH_INFO: `/v3/projects/c04cd6b58f8a4348911e8c14f9fae540` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:21.497138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04cd6b58f8a4348911e8c14f9fae540)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:21.501407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:21.619202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:21.638533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04cd6b58f8a4348911e8c14f9fae540', 'request_id': 'req-d3147935-baec-44ab-af56-c1059ecca0c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:21.719532 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04cd6b58f8a4348911e8c14f9fae540', 'request_id': 'req-d3147935-baec-44ab-af56-c1059ecca0c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:21.731784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04cd6b58f8a4348911e8c14f9fae540', 'request_id': 'req-d3147935-baec-44ab-af56-c1059ecca0c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:21.749846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] The token cache is being invalidate because project c04cd6b58f8a4348911e8c14f9fae540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:21.751857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:21.751857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d3147935-baec-44ab-af56-c1059ecca0c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:21.753131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1033/2050] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:21 2022] DELETE /identity/v3/projects/c04cd6b58f8a4348911e8c14f9fae540 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:21.757786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2315a-e570-4155-8977-7ddf15d1efb7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:21.758280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2315a-e570-4155-8977-7ddf15d1efb7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:21.758379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2315a-e570-4155-8977-7ddf15d1efb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:21.780269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-17a2315a-e570-4155-8977-7ddf15d1efb7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:21.781857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-17a2315a-e570-4155-8977-7ddf15d1efb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:21.821223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:21.858290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1018/2051] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:21.866657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b8c4e-60c3-4802-8498-09a5daac95d6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:21.866943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b8c4e-60c3-4802-8498-09a5daac95d6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:21.867221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b8c4e-60c3-4802-8498-09a5daac95d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:21.897906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-369b8c4e-60c3-4802-8498-09a5daac95d6 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:21.907677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-369b8c4e-60c3-4802-8498-09a5daac95d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:21.932840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1034/2052] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:21.937836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2315a-e570-4155-8977-7ddf15d1efb7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:21.951326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2315a-e570-4155-8977-7ddf15d1efb7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:21.957859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:21.959935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:21.959935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:21.959935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:21.960571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:21.961005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:21.963479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:21.976830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1019/2053] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:21.982052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b8c4e-60c3-4802-8498-09a5daac95d6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:21.993296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b8c4e-60c3-4802-8498-09a5daac95d6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:21.998458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:21.999210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:21.999582 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:22.000117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:22.000841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:22.005639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:22.112790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1035/2054] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:21 2022] POST /identity/v3/projects => generated 361 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:22.121467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:22.147852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88850a0f-98a9-4590-b632-1189464a7fd8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:22.225054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:22.225813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:22.226162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:22.226545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:22.227245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:22.229492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.041073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1020/2055] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:22 2022] POST /identity/v3/users => generated 281 bytes in 917 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:23.042525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.054616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7734e-1a52-4c8c-91a0-5945c38031b4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:23.061671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:23.063482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.063836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.069243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.069243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:23.069243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.079409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1036/2056] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:23.084939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.097601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01c396-98dd-4a92-ae1a-c627cdb91c2d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:23.102818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2104ae4-ea61-4463-b0f7-2db6bde0714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:23.103473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2104ae4-ea61-4463-b0f7-2db6bde0714a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.103788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2104ae4-ea61-4463-b0f7-2db6bde0714a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.104090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2104ae4-ea61-4463-b0f7-2db6bde0714a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.104729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2104ae4-ea61-4463-b0f7-2db6bde0714a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:23.106455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2104ae4-ea61-4463-b0f7-2db6bde0714a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.125798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1021/2057] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:23.126109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.163783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b1f6-dcea-41ae-a99e-9653fe45db26 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:23.169553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:23.172835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.172835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.172835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] PATH_INFO: `/v3/projects/a0516966d1464dc4b80b685980fbc7f2/users/820cbd94896d4b059e319569043cf003/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.201574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0516966d1464dc4b80b685980fbc7f2, user_id=820cbd94896d4b059e319569043cf003, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:23.201574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.313869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1037/2058] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:23 2022] PUT /identity/v3/projects/a0516966d1464dc4b80b685980fbc7f2/users/820cbd94896d4b059e319569043cf003/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:23.326688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.328097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.328097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.365774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] MFA Rules not processed for user `820cbd94896d4b059e319569043cf003`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:23.367260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:23.477118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1022/2059] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:23.482188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.566736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c333935-012b-42e6-9efb-29531adcf0d9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:23.572390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:23.573331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.573821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.574832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.576710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:23.578888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.623738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.625595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:23.649386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274abd4e-2b4d-4a93-8325-215d94240938 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:23.662983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:23.663804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.664179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.664567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] PATH_INFO: `/v3/users/246b8f62c4b94024b9f6b393b15ce67b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.665951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=246b8f62c4b94024b9f6b393b15ce67b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:23.673063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.678327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1038/2060] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:23 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:23.684985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.811675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:23.813740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6193fd-cd3b-4db3-88f7-a8f12340c743 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:23.827281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:23.828009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.828391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.828770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.829529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:23.832075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.913071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:23.941061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246b8f62c4b94024b9f6b393b15ce67b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:23.941558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1039/2061] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:23 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:23.943001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.954433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada76dde-71e7-4258-b9a1-626ee7e58a8b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:23.961706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:23.963130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.963505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.963819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.964667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:23.966643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:23.981818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1040/2062] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:23.984541 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:23.994833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a783b21-6032-4e08-bd69-1cbca3523810 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:24.000806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:24.003217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.003381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.003619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] PATH_INFO: `/v3/projects/8a30f424d6014b718daf513e844d4605/users/2f1628aac2e14e42b153c1fef20625e8/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.006687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246b8f62c4b94024b9f6b393b15ce67b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:24.021917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246b8f62c4b94024b9f6b393b15ce67b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:24.023059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a30f424d6014b718daf513e844d4605, user_id=2f1628aac2e14e42b153c1fef20625e8, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:24.026174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:24.037740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246b8f62c4b94024b9f6b393b15ce67b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:24.038818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ce3d4668-7ad5-432e-b8e0-c0f34aaad40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246b8f62c4b94024b9f6b393b15ce67b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:24.051193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1023/2063] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:23 2022] DELETE /identity/v3/users/246b8f62c4b94024b9f6b393b15ce67b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:24.055452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.055813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.056160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.083028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:24.084258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.140414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1041/2064] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:23 2022] PUT /identity/v3/projects/8a30f424d6014b718daf513e844d4605/users/2f1628aac2e14e42b153c1fef20625e8/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:24.144539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:24.146296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.173993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1024/2065] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:24.185927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:24.187823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.248600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f8962-64b8-420d-9573-38d3d0625037 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:24.256153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:24.258493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.258493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.258493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.258493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:24.260042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:24.281668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1042/2066] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:24.282631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:24.299842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b68d2-8b3e-4c7d-93c0-c07160892f2f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:24.307459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90402b-8a4c-40a3-8b33-8cf3b263dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:24.309795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90402b-8a4c-40a3-8b33-8cf3b263dcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.309795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90402b-8a4c-40a3-8b33-8cf3b263dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.309795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90402b-8a4c-40a3-8b33-8cf3b263dcb4 admin admin] PATH_INFO: `/v3/projects/8a30f424d6014b718daf513e844d4605/users/2f1628aac2e14e42b153c1fef20625e8/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.311525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90402b-8a4c-40a3-8b33-8cf3b263dcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a30f424d6014b718daf513e844d4605, user_id=2f1628aac2e14e42b153c1fef20625e8, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:24.314344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90402b-8a4c-40a3-8b33-8cf3b263dcb4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:24.315953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1440a-3b77-4890-9ca6-5b20652bed22 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:24.338803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1043/2067] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:24 2022] PUT /identity/v3/projects/8a30f424d6014b718daf513e844d4605/users/2f1628aac2e14e42b153c1fef20625e8/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:24.344775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.345076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.345336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.397005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] MFA Rules not processed for user `2f1628aac2e14e42b153c1fef20625e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:24.398115 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.406072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:24.407064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.407579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.408073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.422913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:24.424736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:24.450517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1025/2068] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:24.539110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1044/2069] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:24.553145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:24.554797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.566681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:24.633551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b28b5-0b4c-451f-ad48-44dd46ba1f34 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:24.729546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:24.730161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.731059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.731477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.749831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:24.751707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:24.764397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1026/2070] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:25.007283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:25.022185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:25.040510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:25.124213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accf9d0-f81e-47bc-b16a-201bd937eab9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:25.129577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:25.130429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:25.130576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:25.130850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] PATH_INFO: `/v3/projects/a69bcd133a2b42309e71a5b96ee99add` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:25.131986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69bcd133a2b42309e71a5b96ee99add)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:25.133962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:25.256000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69bcd133a2b42309e71a5b96ee99add', 'request_id': 'req-e92f7788-da22-4081-8da7-7c8f2d95b3de'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.333073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69bcd133a2b42309e71a5b96ee99add', 'request_id': 'req-e92f7788-da22-4081-8da7-7c8f2d95b3de'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.342530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69bcd133a2b42309e71a5b96ee99add', 'request_id': 'req-e92f7788-da22-4081-8da7-7c8f2d95b3de'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.357368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] The token cache is being invalidate because project a69bcd133a2b42309e71a5b96ee99add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:25.357600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.358374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.360728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1045/2071] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:25 2022] DELETE /identity/v3/projects/a69bcd133a2b42309e71a5b96ee99add => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:25.365268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:25.388734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d9de6-a01d-4b13-89ec-9d6620618722 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:25.470514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:25.471729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:25.471937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:25.472217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] PATH_INFO: `/v3/users/12b8f31f6da44e71adf8a5906e1d96dc` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:25.473796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b8f31f6da44e71adf8a5906e1d96dc)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:25.475800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:25.755320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b8f31f6da44e71adf8a5906e1d96dc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.846067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b8f31f6da44e71adf8a5906e1d96dc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.857613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:25.860012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b8f31f6da44e71adf8a5906e1d96dc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.869914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b8f31f6da44e71adf8a5906e1d96dc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.870746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d9233b-2347-46c3-b8e1-303471d5247b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b8f31f6da44e71adf8a5906e1d96dc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:25.877516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1027/2072] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:25 2022] DELETE /identity/v3/users/12b8f31f6da44e71adf8a5906e1d96dc => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:25.928066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:25.929569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:25.963523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f7788-da22-4081-8da7-7c8f2d95b3de admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:26.064084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:26.065058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:26.065506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:26.065947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] PATH_INFO: `/v3/users/820cbd94896d4b059e319569043cf003` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:26.067483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820cbd94896d4b059e319569043cf003)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:26.070934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:26.266306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:26.268859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820cbd94896d4b059e319569043cf003'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:26.330817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820cbd94896d4b059e319569043cf003'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:26.345458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820cbd94896d4b059e319569043cf003'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:26.346533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820cbd94896d4b059e319569043cf003'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:26.347980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820cbd94896d4b059e319569043cf003'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:26.375676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1046/2073] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:25 2022] DELETE /identity/v3/users/820cbd94896d4b059e319569043cf003 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:26.379982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:26.380538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:26.380538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:26.403623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:26.404949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:26.487208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1028/2074] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:26.514033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:26.514033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:26.537973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:26.541223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:26.556259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b866d2-49bb-446c-955e-101e04a47d80 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:26.563211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:26.563851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:26.564136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:26.564409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] PATH_INFO: `/v3/projects/701e09d0a5184a6a9cc85f6e6d225c3b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:26.566454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=701e09d0a5184a6a9cc85f6e6d225c3b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:26.568280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:26.638791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b5cb-cb81-42d1-9e9a-9946dddd5ac6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:26.642484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d8210-bf36-47a7-a4c8-08aa0280e181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:26.643083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d8210-bf36-47a7-a4c8-08aa0280e181 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:26.643361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d8210-bf36-47a7-a4c8-08aa0280e181 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:26.643628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d8210-bf36-47a7-a4c8-08aa0280e181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:26.653669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d8210-bf36-47a7-a4c8-08aa0280e181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:26.655145 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d8210-bf36-47a7-a4c8-08aa0280e181 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:26.665257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1047/2075] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:27.107042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701e09d0a5184a6a9cc85f6e6d225c3b', 'request_id': 'req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.115418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701e09d0a5184a6a9cc85f6e6d225c3b', 'request_id': 'req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.201052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701e09d0a5184a6a9cc85f6e6d225c3b', 'request_id': 'req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.219280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] The token cache is being invalidate because project 701e09d0a5184a6a9cc85f6e6d225c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:27.219618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.220360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0ef6d8cf-bf4c-44d2-b909-329a92e581f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.227122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.227431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.227669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.236764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1029/2076] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:26 2022] DELETE /identity/v3/projects/701e09d0a5184a6a9cc85f6e6d225c3b => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:27.258791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:27.262622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:27.354615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1048/2077] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:27.358787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cab7fe-623a-4c4d-9fdb-0443b173d034 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.359144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cab7fe-623a-4c4d-9fdb-0443b173d034 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.359487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cab7fe-623a-4c4d-9fdb-0443b173d034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.384903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c5cab7fe-623a-4c4d-9fdb-0443b173d034 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:27.387301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c5cab7fe-623a-4c4d-9fdb-0443b173d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:27.395716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:27.400223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:27.402180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1030/2078] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:27.408114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cab7fe-623a-4c4d-9fdb-0443b173d034 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:27.427578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cab7fe-623a-4c4d-9fdb-0443b173d034 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:27.438932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0329f-5f9c-4930-b124-484acd1954a1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:27.441941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:27.442696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.443069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.443441 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.444412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:27.444886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:27.448154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:27.449580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.449920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:27.450045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.450374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] PATH_INFO: `/v3/projects/a0516966d1464dc4b80b685980fbc7f2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.451590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0516966d1464dc4b80b685980fbc7f2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:27.454129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:27.478082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1031/2079] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:27.482966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:27.494170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1a206-6975-44f0-9612-12cf1af517c4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:27.500405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:27.501208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.501488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.501759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.502409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:27.504989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:27.611352 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0516966d1464dc4b80b685980fbc7f2', 'request_id': 'req-6fa9525b-d029-4972-b7bc-d7fabbea64bc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.756876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1032/2080] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:27 2022] POST /identity/v3/projects => generated 387 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:27.761087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:27.789465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:27.811603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0516966d1464dc4b80b685980fbc7f2', 'request_id': 'req-6fa9525b-d029-4972-b7bc-d7fabbea64bc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.819267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0516966d1464dc4b80b685980fbc7f2', 'request_id': 'req-6fa9525b-d029-4972-b7bc-d7fabbea64bc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.820772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:27.823120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35b172-17bd-4380-8c65-f6bb3fd6519f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:27.828604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:27.829211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.829553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.829830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.830552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:27.832783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:27.843411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] The token cache is being invalidate because project a0516966d1464dc4b80b685980fbc7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:27.843965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.844971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:27.846650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1049/2081] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:27 2022] DELETE /identity/v3/projects/a0516966d1464dc4b80b685980fbc7f2 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:27.852540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:27.852540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:27.915847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9525b-d029-4972-b7bc-d7fabbea64bc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:27.925284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:27.925284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.925284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.925284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] PATH_INFO: `/v3/users/2f1628aac2e14e42b153c1fef20625e8` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.925817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1628aac2e14e42b153c1fef20625e8)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:27.927685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:27.961048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1033/2082] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:27 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:27.965542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:27.980004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:27.988476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07300c-ceac-4fd9-8dc3-fe27a87d7337 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:27.993160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:27.993758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.994037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.994307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.994991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:27.997592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.012513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1034/2083] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:28.013601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.023402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8850def-605c-47df-b5ce-6351c8c59fac admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.029033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.031249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.031249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.031249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.031249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.032527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.042948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1035/2084] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:28.047502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.060240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0458c-dba9-48de-9da1-02c9dcd82859 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.062339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae300-6fd3-471e-b80e-49c9aa4aafe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.063017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae300-6fd3-471e-b80e-49c9aa4aafe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.063295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae300-6fd3-471e-b80e-49c9aa4aafe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.063568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae300-6fd3-471e-b80e-49c9aa4aafe2 admin admin] PATH_INFO: `/v3/projects/cc11e00341664731b5a6223ee217d257/users/f5d25d7504a54206a717505606b2d2f2/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.078062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae300-6fd3-471e-b80e-49c9aa4aafe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc11e00341664731b5a6223ee217d257, user_id=f5d25d7504a54206a717505606b2d2f2, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.080824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae300-6fd3-471e-b80e-49c9aa4aafe2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.150272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1628aac2e14e42b153c1fef20625e8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.182329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1036/2085] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:28 2022] PUT /identity/v3/projects/cc11e00341664731b5a6223ee217d257/users/f5d25d7504a54206a717505606b2d2f2/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:28.194082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.194509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.194791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.223375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] MFA Rules not processed for user `f5d25d7504a54206a717505606b2d2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:28.224656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:28.252002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1628aac2e14e42b153c1fef20625e8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.266918 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1628aac2e14e42b153c1fef20625e8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.267776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1628aac2e14e42b153c1fef20625e8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.268404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f2ccf8c4-acd7-4132-9b01-6286e1717c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1628aac2e14e42b153c1fef20625e8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.295804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1037/2086] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:28.300214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.301508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:28.304995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1050/2087] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:27 2022] DELETE /identity/v3/users/2f1628aac2e14e42b153c1fef20625e8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:28.352274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b003bd4-d181-4a0d-83f2-5fcdca57d46f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.352735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b003bd4-d181-4a0d-83f2-5fcdca57d46f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.353249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b003bd4-d181-4a0d-83f2-5fcdca57d46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.377433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6b003bd4-d181-4a0d-83f2-5fcdca57d46f None None] MFA Rules not processed for user `747b60cf32a14b93bee399fce7a9b54b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:28.378832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6b003bd4-d181-4a0d-83f2-5fcdca57d46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:28.393413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028e84f-3527-438e-8571-ad300c2ab789 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.436852 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1051/2088] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:28.452996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.453641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.453907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.454218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.454880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.457020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.503386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:28.569630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1038/2089] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:28 2022] POST /identity/v3/projects => generated 387 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:28.573457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b003bd4-d181-4a0d-83f2-5fcdca57d46f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.586819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b003bd4-d181-4a0d-83f2-5fcdca57d46f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.640837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:28.657682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.658849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.659130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.659400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.660034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.662019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.779444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1052/2090] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:28 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:28.782934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.784672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:28.787692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.798166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35879-1362-40e4-86a7-3cbe876bf055 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.803871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.804673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.805053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.805431 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.806877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.808600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.820317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1053/2091] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:28.827853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.837066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7574fd-82a9-4c8b-8f2f-2fb603328967 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.839897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fef136-308f-4fd4-9fea-96b33df25df3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.842624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.843268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.843555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.843826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] PATH_INFO: `/v3/projects/8a30f424d6014b718daf513e844d4605` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.845212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a30f424d6014b718daf513e844d4605)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.846167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.846937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.847203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.848069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.848504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] PATH_INFO: `/v3/projects/f6c9f05cf6ad4e269dfda69bfcb50e6e/users/6fc9e35ff9d241f7a34b5d2a04bb82c9/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.867278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c9f05cf6ad4e269dfda69bfcb50e6e, user_id=6fc9e35ff9d241f7a34b5d2a04bb82c9, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.870261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.934341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1054/2092] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:28 2022] PUT /identity/v3/projects/f6c9f05cf6ad4e269dfda69bfcb50e6e/users/6fc9e35ff9d241f7a34b5d2a04bb82c9/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:28.938435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.989796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a30f424d6014b718daf513e844d4605', 'request_id': 'req-0260c2f8-a0f1-4553-8595-9036f1a0462e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.998561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a30f424d6014b718daf513e844d4605', 'request_id': 'req-0260c2f8-a0f1-4553-8595-9036f1a0462e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.012611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90bee0-b083-47ed-ad6d-7172c6317c17 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.070472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.071073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.071354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.071624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.072264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.073841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.084603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1055/2093] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:29.088580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.097452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a30f424d6014b718daf513e844d4605', 'request_id': 'req-0260c2f8-a0f1-4553-8595-9036f1a0462e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.098617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce4f79-3dc8-4c77-aa50-9ca32f81271e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.103590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc695f1-773c-4941-8c26-30d8bfd3e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.112781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc695f1-773c-4941-8c26-30d8bfd3e472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.112781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc695f1-773c-4941-8c26-30d8bfd3e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.112781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc695f1-773c-4941-8c26-30d8bfd3e472 admin admin] PATH_INFO: `/v3/projects/f6c9f05cf6ad4e269dfda69bfcb50e6e/users/6fc9e35ff9d241f7a34b5d2a04bb82c9/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.112781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc695f1-773c-4941-8c26-30d8bfd3e472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c9f05cf6ad4e269dfda69bfcb50e6e, user_id=6fc9e35ff9d241f7a34b5d2a04bb82c9, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.112781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc695f1-773c-4941-8c26-30d8bfd3e472 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.113660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] The token cache is being invalidate because project 8a30f424d6014b718daf513e844d4605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:29.113660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.113660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0260c2f8-a0f1-4553-8595-9036f1a0462e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.113660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1039/2094] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:28 2022] DELETE /identity/v3/projects/8a30f424d6014b718daf513e844d4605 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:29.117382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18f19-d9c0-4abe-935f-299401a455be None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.117668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18f19-d9c0-4abe-935f-299401a455be None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.117946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc18f19-d9c0-4abe-935f-299401a455be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.123339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1056/2095] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:29 2022] PUT /identity/v3/projects/f6c9f05cf6ad4e269dfda69bfcb50e6e/users/6fc9e35ff9d241f7a34b5d2a04bb82c9/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:29.133501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.133979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.134437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.142601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2fc18f19-d9c0-4abe-935f-299401a455be None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:29.143929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2fc18f19-d9c0-4abe-935f-299401a455be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.178096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] MFA Rules not processed for user `6fc9e35ff9d241f7a34b5d2a04bb82c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:29.179594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.234512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1040/2096] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:29.240256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921b83-91fd-439a-a2a0-b6008484c4d8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.242782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921b83-91fd-439a-a2a0-b6008484c4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.243069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921b83-91fd-439a-a2a0-b6008484c4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.274193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d2921b83-91fd-439a-a2a0-b6008484c4d8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:29.280008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d2921b83-91fd-439a-a2a0-b6008484c4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.280157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1057/2097] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:29.284502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1041/2098] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:29.287146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:29.288626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.289067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921b83-91fd-439a-a2a0-b6008484c4d8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.290421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.300874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921b83-91fd-439a-a2a0-b6008484c4d8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.306848 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.307596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.307944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.308284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.309906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:29.310281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.312142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.315429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:29.327999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1042/2099] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:29.345955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.367740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d4d36-8836-4970-8d65-666d74b03b43 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.373928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.379556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.379556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.379556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.379556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.379912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbafe8f-2e09-4f96-b633-720c2fd7648d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.380353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.382081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.382900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.383282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.383657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.395580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.397868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.409984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1058/2100] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:29.491946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.497602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.512488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1043/2101] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:29 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:29.516454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.516714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965e692-2703-4c63-9b9c-ec6650e4b585 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.583011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241786f1-e2e9-4e14-bfbf-7848919b6f6f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.583163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.583822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.584234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.584658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] PATH_INFO: `/v3/users/f5d25d7504a54206a717505606b2d2f2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.585901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d25d7504a54206a717505606b2d2f2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.588137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.589280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.590198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.591107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.591266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.597471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.597471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.706884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1044/2102] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:29 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:29.710860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.721025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c60ed-84ee-43f2-b4b4-9c286b747f7c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.725928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.726566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.726847 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.727228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.727932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.729515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.740311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1045/2103] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:29.744261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.754290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84b410-b09a-4628-9969-7a562d39fdba admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.758979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.759564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.764110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.764110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.764110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.764110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.769292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d25d7504a54206a717505606b2d2f2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.773337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1046/2104] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:29.777703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.787278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f635e96-81e2-4550-b246-18b9a8db48a7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.790981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:29.794345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e4f09-7ff0-439c-8656-1d047c833113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.799015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e4f09-7ff0-439c-8656-1d047c833113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.799015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e4f09-7ff0-439c-8656-1d047c833113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.799015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e4f09-7ff0-439c-8656-1d047c833113 admin admin] PATH_INFO: `/v3/projects/e431f63014204603ab0dba1d4fe646f4/users/08ff23d43abb4c128a3c00a9654d552d/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.817885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e4f09-7ff0-439c-8656-1d047c833113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e431f63014204603ab0dba1d4fe646f4, user_id=08ff23d43abb4c128a3c00a9654d552d, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.820850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e4f09-7ff0-439c-8656-1d047c833113 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.851568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d25d7504a54206a717505606b2d2f2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.863970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d25d7504a54206a717505606b2d2f2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.864625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d25d7504a54206a717505606b2d2f2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.865251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-773dd86e-67f5-48fa-bd71-d6e8f132aaef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d25d7504a54206a717505606b2d2f2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.877899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1059/2105] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:29 2022] DELETE /identity/v3/users/f5d25d7504a54206a717505606b2d2f2 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:29.882627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.891834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.891834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.903610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:29.904855 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.922165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1047/2106] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:29 2022] PUT /identity/v3/projects/e431f63014204603ab0dba1d4fe646f4/users/08ff23d43abb4c128a3c00a9654d552d/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:29.928533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.928897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.929194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.984373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] MFA Rules not processed for user `08ff23d43abb4c128a3c00a9654d552d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:29.985596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.991293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1060/2107] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:30.003547 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.005271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:30.061798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1048/2108] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:30.066112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.067569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:30.075410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58dcec9-dbcb-46e8-b848-4d362dffcb2f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.080330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.081114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.081702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.082084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.086730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec249c-5c09-4070-b2e9-5d2f6c19c381 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.093710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.094203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.094203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.094203 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.094708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.094801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.095294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.097253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.114442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1061/2109] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:30.153145 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:30.193991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1049/2110] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:30 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:30.197676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.207955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63dbe72-be88-4a4c-912a-2a3f40cfb2ad service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.270047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.270799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.271151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.271576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.272346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.274370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.327055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:30.366409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1062/2111] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:30 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:30.372393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.383946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc815c-b261-4160-a8f8-e3503f0e9191 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.390924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.392209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.392490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.392761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.396562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.396562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.408360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1050/2112] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:30.412355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.423642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617c765-ae03-47c2-be5c-81a5a89c5854 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.429152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.429344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.429623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.429927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.430606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.432211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.442411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1063/2113] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:30.446927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.459260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec4945-0647-4b1e-a187-1fcea9496a55 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.464018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bdd57-ec07-452e-9951-d2f6bf8ef248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.464671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdd57-ec07-452e-9951-d2f6bf8ef248 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.465494 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdd57-ec07-452e-9951-d2f6bf8ef248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.465854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdd57-ec07-452e-9951-d2f6bf8ef248 admin admin] PATH_INFO: `/v3/projects/5b94911b4ab94a26a28f37cc8fdf9ea0/users/d1c7a0e1c87a4f1591ad9d22e8266d72/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.487427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bdd57-ec07-452e-9951-d2f6bf8ef248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b94911b4ab94a26a28f37cc8fdf9ea0, user_id=d1c7a0e1c87a4f1591ad9d22e8266d72, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.490336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bdd57-ec07-452e-9951-d2f6bf8ef248 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.586252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1051/2114] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:30 2022] PUT /identity/v3/projects/5b94911b4ab94a26a28f37cc8fdf9ea0/users/d1c7a0e1c87a4f1591ad9d22e8266d72/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:30.586816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.586816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:30.594094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8314b13-aede-4f23-bf2a-34f3d09b94b9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.594340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8314b13-aede-4f23-bf2a-34f3d09b94b9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.596816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8314b13-aede-4f23-bf2a-34f3d09b94b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.604480 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b941c-a3f1-4e08-b106-af4b8c59ab43 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.639071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c8314b13-aede-4f23-bf2a-34f3d09b94b9 None None] MFA Rules not processed for user `d1c7a0e1c87a4f1591ad9d22e8266d72`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:30.640333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c8314b13-aede-4f23-bf2a-34f3d09b94b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:30.723321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.725858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.725858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.725858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] PATH_INFO: `/v3/projects/cc11e00341664731b5a6223ee217d257` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.725858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc11e00341664731b5a6223ee217d257)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.728615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.769478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1052/2115] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:30.770644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8314b13-aede-4f23-bf2a-34f3d09b94b9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.784319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8314b13-aede-4f23-bf2a-34f3d09b94b9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.789238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.789956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.790377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.790838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.791576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.793818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.867848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc11e00341664731b5a6223ee217d257', 'request_id': 'req-3842be57-9956-4cfb-86ea-876a46c0c9fe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.883340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1053/2116] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:30 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:30.887699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.957235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd66b-bb0f-4ae6-920e-1d6013cbc090 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.961978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.962541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.962801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.963069 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.963672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.966242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.966821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc11e00341664731b5a6223ee217d257', 'request_id': 'req-3842be57-9956-4cfb-86ea-876a46c0c9fe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.975544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc11e00341664731b5a6223ee217d257', 'request_id': 'req-3842be57-9956-4cfb-86ea-876a46c0c9fe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.989638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] The token cache is being invalidate because project cc11e00341664731b5a6223ee217d257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:30.990377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.991147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.993047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1064/2117] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:30 2022] DELETE /identity/v3/projects/cc11e00341664731b5a6223ee217d257 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:30.996623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.997982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.051920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1054/2118] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:30 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:34:31.055903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.057583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.070609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5469a17-e702-4d0e-a57a-0c1ff0ac78d4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.071008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:31.075689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842be57-9956-4cfb-86ea-876a46c0c9fe admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.076164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.076817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.077097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.077364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.077994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.079637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.081682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:31.083383 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.084196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.084478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.084757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] PATH_INFO: `/v3/users/6fc9e35ff9d241f7a34b5d2a04bb82c9` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.086691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fc9e35ff9d241f7a34b5d2a04bb82c9)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.089243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.091125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1055/2119] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:31.095082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.105521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350e61e-943c-423a-9f78-82ee6ed2e575 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.114989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.115732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.116076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.116410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] PATH_INFO: `/v3/projects/f4f13e6bb10f4b12b10d68091d18f8aa/users/6c4120da1223409ebcc9954768d35180/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.132780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f13e6bb10f4b12b10d68091d18f8aa, user_id=6c4120da1223409ebcc9954768d35180, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.146216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.217647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1056/2120] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:31 2022] PUT /identity/v3/projects/f4f13e6bb10f4b12b10d68091d18f8aa/users/6c4120da1223409ebcc9954768d35180/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:31.221410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.282270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13b053-75ef-4975-932d-723baa93dec5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.287370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.288110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.288396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.288669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.292299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.292299 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.299718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fc9e35ff9d241f7a34b5d2a04bb82c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.301502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1057/2121] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:31.305543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.315739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a43f5a-421a-4f09-8123-aa50d740114c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.320371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab82641-8526-4c17-82b2-24ada4861ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.320988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab82641-8526-4c17-82b2-24ada4861ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.321268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab82641-8526-4c17-82b2-24ada4861ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.321538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab82641-8526-4c17-82b2-24ada4861ab6 admin admin] PATH_INFO: `/v3/projects/f4f13e6bb10f4b12b10d68091d18f8aa/users/6c4120da1223409ebcc9954768d35180/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.323463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab82641-8526-4c17-82b2-24ada4861ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f13e6bb10f4b12b10d68091d18f8aa, user_id=6c4120da1223409ebcc9954768d35180, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.326786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab82641-8526-4c17-82b2-24ada4861ab6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.336737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1058/2122] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:31 2022] PUT /identity/v3/projects/f4f13e6bb10f4b12b10d68091d18f8aa/users/6c4120da1223409ebcc9954768d35180/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:31.349190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.349614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.350065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.380229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] MFA Rules not processed for user `6c4120da1223409ebcc9954768d35180`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:31.381425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.389853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fc9e35ff9d241f7a34b5d2a04bb82c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.403691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fc9e35ff9d241f7a34b5d2a04bb82c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.404438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fc9e35ff9d241f7a34b5d2a04bb82c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.405039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fc9e35ff9d241f7a34b5d2a04bb82c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.419296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1065/2123] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:30 2022] DELETE /identity/v3/users/6fc9e35ff9d241f7a34b5d2a04bb82c9 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:31.459342 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1059/2124] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:31.471935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.474644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.536860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5ec51-872d-4b91-b8af-fcd2411715b6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.540492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd992329-44f7-4799-a6d0-02d12912a39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.545405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992329-44f7-4799-a6d0-02d12912a39c service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.545405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992329-44f7-4799-a6d0-02d12912a39c service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.545405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992329-44f7-4799-a6d0-02d12912a39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.602799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd992329-44f7-4799-a6d0-02d12912a39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.604215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd992329-44f7-4799-a6d0-02d12912a39c service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.614246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1066/2125] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:32.007538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.011330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.070108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:32.101114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c97c3d-b903-482a-8c40-64fc6c455608 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.162302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.163043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.163317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.163586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] PATH_INFO: `/v3/projects/f6c9f05cf6ad4e269dfda69bfcb50e6e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.164756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c9f05cf6ad4e269dfda69bfcb50e6e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.166903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.269190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c9f05cf6ad4e269dfda69bfcb50e6e', 'request_id': 'req-98d13dc5-0be9-459f-9260-5853ed9eea99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.275852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c9f05cf6ad4e269dfda69bfcb50e6e', 'request_id': 'req-98d13dc5-0be9-459f-9260-5853ed9eea99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.338028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c9f05cf6ad4e269dfda69bfcb50e6e', 'request_id': 'req-98d13dc5-0be9-459f-9260-5853ed9eea99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.351699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] The token cache is being invalidate because project f6c9f05cf6ad4e269dfda69bfcb50e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:32.351982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.352607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-98d13dc5-0be9-459f-9260-5853ed9eea99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.354079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1060/2126] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:32 2022] DELETE /identity/v3/projects/f6c9f05cf6ad4e269dfda69bfcb50e6e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:32.358970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7c69d-f712-496d-adb7-d7e991bbc5a5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.359260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7c69d-f712-496d-adb7-d7e991bbc5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.359537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7c69d-f712-496d-adb7-d7e991bbc5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.380463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2aa7c69d-f712-496d-adb7-d7e991bbc5a5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:32.381679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2aa7c69d-f712-496d-adb7-d7e991bbc5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.402230 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:32.444783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1067/2127] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:32.453656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5be0a-f6b8-4209-b2a0-89648455b99b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.453983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5be0a-f6b8-4209-b2a0-89648455b99b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.454260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5be0a-f6b8-4209-b2a0-89648455b99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.475115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b9d5be0a-f6b8-4209-b2a0-89648455b99b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:32.476314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b9d5be0a-f6b8-4209-b2a0-89648455b99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.484495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1061/2128] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:32.488598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7c69d-f712-496d-adb7-d7e991bbc5a5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.499178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7c69d-f712-496d-adb7-d7e991bbc5a5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.504569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.505198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.505531 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.505810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.506518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:32.506820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.508440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.521328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1068/2129] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:32.525616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5be0a-f6b8-4209-b2a0-89648455b99b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.535730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5be0a-f6b8-4209-b2a0-89648455b99b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.540456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.541088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.541365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.541635 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.542262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.544372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.639044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1062/2130] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:32 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:32.641896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.652060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a6ec-136a-4182-93ff-987054ddd59e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.712213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.712803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.713079 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.713344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.714217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.716163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.736584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.740476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.790008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:32.793781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbc6b1-06e5-4f98-b710-e55806952248 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.797417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.798732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.798732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.798732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.800355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.827345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1069/2131] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:32 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:32.828341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.838183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40828b-1567-446b-ab40-7ac4d5da8d90 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.843082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.843668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.843944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.844304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.844842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.846441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.860156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1070/2132] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:32.860340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.861183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.861295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.870821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22deef1-1b4c-4f7b-8637-3ad7f64e71d1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.872332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1063/2133] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:32.876311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db47188-b347-4af5-8d71-7f791690522e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.876891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db47188-b347-4af5-8d71-7f791690522e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.877171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db47188-b347-4af5-8d71-7f791690522e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.877440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db47188-b347-4af5-8d71-7f791690522e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.878072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db47188-b347-4af5-8d71-7f791690522e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.879666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db47188-b347-4af5-8d71-7f791690522e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.890338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1071/2134] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:32.894755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.906345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05625f-580d-48f9-a48a-78cd92c99e30 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.913244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.913750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.914460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.914806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] PATH_INFO: `/v3/projects/caf68ec4bd4c463fb6a196a8695184a1/users/3d8f5ac0c55f4c3c93e66e6e992fd984/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.935161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf68ec4bd4c463fb6a196a8695184a1, user_id=3d8f5ac0c55f4c3c93e66e6e992fd984, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.938118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:33.017036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1064/2135] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:32 2022] PUT /identity/v3/projects/caf68ec4bd4c463fb6a196a8695184a1/users/3d8f5ac0c55f4c3c93e66e6e992fd984/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:33.027393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76768c-9236-44b1-8e5b-a2e1c00d20a6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.027896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76768c-9236-44b1-8e5b-a2e1c00d20a6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.028220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76768c-9236-44b1-8e5b-a2e1c00d20a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.058008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4a76768c-9236-44b1-8e5b-a2e1c00d20a6 None None] MFA Rules not processed for user `3d8f5ac0c55f4c3c93e66e6e992fd984`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:33.061370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4a76768c-9236-44b1-8e5b-a2e1c00d20a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:33.142499 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1072/2136] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:33.148004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:33.223221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e6b5a-e72a-4aca-8bdc-88292f88403c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:33.228282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:33.228966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.229250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.229652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.230845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:33.232863 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:33.382866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1065/2137] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:33 2022] POST /identity/v3/projects => generated 389 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:33.384398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76768c-9236-44b1-8e5b-a2e1c00d20a6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:33.396207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76768c-9236-44b1-8e5b-a2e1c00d20a6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:33.405919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:33.454935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:33.460218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.460218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.460218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.460218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:33.460218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:33.551299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1073/2138] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:33 2022] POST /identity/v3/users => generated 301 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:33.555153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:33.568019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f42d6-c832-4921-9e57-13bf0b94e00a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:33.573026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:33.573619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.573890 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.574376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.575468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:33.577528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:33.589188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1066/2139] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:33.593873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:33.606158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5903d6-c46e-4052-9147-8427d6a50132 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:33.611311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:33.611975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.612268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.612533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] PATH_INFO: `/v3/projects/d58b9ee1df8a4d2dbcb7a6f98c3e4469/users/3ac274ad21e1427ab1b88f40f527fc0e/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.643010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58b9ee1df8a4d2dbcb7a6f98c3e4469, user_id=3ac274ad21e1427ab1b88f40f527fc0e, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:33.645658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:33.761396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1074/2140] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:33 2022] PUT /identity/v3/projects/d58b9ee1df8a4d2dbcb7a6f98c3e4469/users/3ac274ad21e1427ab1b88f40f527fc0e/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:33.765983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:33.776330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1645c13-3fea-404c-8efc-03b9e9dcd395 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:33.829336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:33.838732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de661e9-85cf-48f9-b754-aa0ddcaab4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:33.844234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de661e9-85cf-48f9-b754-aa0ddcaab4ec admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.844234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de661e9-85cf-48f9-b754-aa0ddcaab4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.844234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de661e9-85cf-48f9-b754-aa0ddcaab4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.844234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de661e9-85cf-48f9-b754-aa0ddcaab4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:33.844234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de661e9-85cf-48f9-b754-aa0ddcaab4ec admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:33.856382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1067/2141] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:33.860682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:33.873941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daea511-63f7-41c9-8c08-d20d77442850 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:33.879310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b251d-c2da-4faa-bd21-7b870818e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:33.879969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b251d-c2da-4faa-bd21-7b870818e2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.880275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b251d-c2da-4faa-bd21-7b870818e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.880588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b251d-c2da-4faa-bd21-7b870818e2c4 admin admin] PATH_INFO: `/v3/projects/d58b9ee1df8a4d2dbcb7a6f98c3e4469/users/3ac274ad21e1427ab1b88f40f527fc0e/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.882605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b251d-c2da-4faa-bd21-7b870818e2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58b9ee1df8a4d2dbcb7a6f98c3e4469, user_id=3ac274ad21e1427ab1b88f40f527fc0e, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:33.885622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b251d-c2da-4faa-bd21-7b870818e2c4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:33.898075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1075/2142] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:33 2022] PUT /identity/v3/projects/d58b9ee1df8a4d2dbcb7a6f98c3e4469/users/3ac274ad21e1427ab1b88f40f527fc0e/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:33.908948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89953a34-ace5-4cbf-98b4-232163c02d11 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.909238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89953a34-ace5-4cbf-98b4-232163c02d11 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.909513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89953a34-ace5-4cbf-98b4-232163c02d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:33.939448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-89953a34-ace5-4cbf-98b4-232163c02d11 None None] MFA Rules not processed for user `3ac274ad21e1427ab1b88f40f527fc0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:33.940780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-89953a34-ace5-4cbf-98b4-232163c02d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:33.988895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:33.989916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:33.989916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:34.010436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:34.012481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:34.024877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1068/2143] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:33 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:34.038990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89953a34-ace5-4cbf-98b4-232163c02d11 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:34.102694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1076/2144] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:34:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:34.112656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89953a34-ace5-4cbf-98b4-232163c02d11 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:34.114211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:34.115699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:34.116067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:34.116683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:34.116961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:34.117231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:34.128948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:34.130540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:34.153249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1069/2145] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:34.208261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95003a9d-88ed-4531-845f-de61616a6e22 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:34.224809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547df182-3cd7-48c4-ab07-920f04b2fa2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:34.225505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df182-3cd7-48c4-ab07-920f04b2fa2a service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:34.225859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df182-3cd7-48c4-ab07-920f04b2fa2a service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:34.226261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df182-3cd7-48c4-ab07-920f04b2fa2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:34.241086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547df182-3cd7-48c4-ab07-920f04b2fa2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:34.249258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547df182-3cd7-48c4-ab07-920f04b2fa2a service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:34.253374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:34.264126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1077/2146] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:34.358655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:34.432255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a755da-fe3b-4f02-b20b-c4a7a7336424 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:34.440784 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:34.441424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:34.441695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:34.441968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] PATH_INFO: `/v3/users/3d8f5ac0c55f4c3c93e66e6e992fd984` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:34.443180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8f5ac0c55f4c3c93e66e6e992fd984)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:34.445121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:34.508941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:34.773007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8f5ac0c55f4c3c93e66e6e992fd984'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:34.835460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8f5ac0c55f4c3c93e66e6e992fd984'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:34.854444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8f5ac0c55f4c3c93e66e6e992fd984'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:34.872049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8f5ac0c55f4c3c93e66e6e992fd984'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:34.872049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8f5ac0c55f4c3c93e66e6e992fd984'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:34.874341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1070/2147] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:34 2022] DELETE /identity/v3/users/3d8f5ac0c55f4c3c93e66e6e992fd984 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:34.879109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:34.879917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:34.880357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:34.906982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:34.915380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:35.011384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1078/2148] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:35.023057 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:35.025348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:35.151722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33a59-9c74-48a9-84bc-eafe8e875572 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:35.178190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:35.179121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:35.179571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:35.180106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:35.211289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:35.212749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:35.224431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1071/2149] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:35.871143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:35.872749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:35.885836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621a070-1818-420a-bf3f-9491c5b499cf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:35.893076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:35.893716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:35.893993 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:35.894269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] PATH_INFO: `/v3/projects/caf68ec4bd4c463fb6a196a8695184a1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:35.895608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf68ec4bd4c463fb6a196a8695184a1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:35.897670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:36.048276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf68ec4bd4c463fb6a196a8695184a1', 'request_id': 'req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.135611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf68ec4bd4c463fb6a196a8695184a1', 'request_id': 'req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.144155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf68ec4bd4c463fb6a196a8695184a1', 'request_id': 'req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.172249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] The token cache is being invalidate because project caf68ec4bd4c463fb6a196a8695184a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:36.172466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.173136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.174754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1079/2150] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:35 2022] DELETE /identity/v3/projects/caf68ec4bd4c463fb6a196a8695184a1 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:36.178970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:36.188836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bc217-3895-404e-8e4b-9996733bff17 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:36.257116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:36.257955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:36.258079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:36.258313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] PATH_INFO: `/v3/users/3ac274ad21e1427ab1b88f40f527fc0e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:36.259661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac274ad21e1427ab1b88f40f527fc0e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:36.262174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:36.480880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:36.484568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac274ad21e1427ab1b88f40f527fc0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.582384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac274ad21e1427ab1b88f40f527fc0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.608270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac274ad21e1427ab1b88f40f527fc0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.626369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac274ad21e1427ab1b88f40f527fc0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.632385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-768c8e7c-419e-4cf0-8a14-55bc33b8c789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac274ad21e1427ab1b88f40f527fc0e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:36.656799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1072/2151] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:36 2022] DELETE /identity/v3/users/3ac274ad21e1427ab1b88f40f527fc0e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:37.302219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:37.303962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:37.331591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f62ab3-5750-4b8f-8647-6cf5e6a66a3b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:37.439363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:37.440692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:37.442393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:37.442700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] PATH_INFO: `/v3/projects/d58b9ee1df8a4d2dbcb7a6f98c3e4469` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:37.444960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58b9ee1df8a4d2dbcb7a6f98c3e4469)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:37.451789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:37.515472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:37.603164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58b9ee1df8a4d2dbcb7a6f98c3e4469', 'request_id': 'req-a242a7b5-41a7-49ba-97fb-1185aa15a594'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:37.741573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58b9ee1df8a4d2dbcb7a6f98c3e4469', 'request_id': 'req-a242a7b5-41a7-49ba-97fb-1185aa15a594'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:37.749835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58b9ee1df8a4d2dbcb7a6f98c3e4469', 'request_id': 'req-a242a7b5-41a7-49ba-97fb-1185aa15a594'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:37.761026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] The token cache is being invalidate because project d58b9ee1df8a4d2dbcb7a6f98c3e4469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:37.761459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:37.762961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a242a7b5-41a7-49ba-97fb-1185aa15a594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:37.764922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1080/2152] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:37 2022] DELETE /identity/v3/projects/d58b9ee1df8a4d2dbcb7a6f98c3e4469 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:37.769741 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f1fe1-b58d-4ca1-b979-bc4d2836d225 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:37.772712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f1fe1-b58d-4ca1-b979-bc4d2836d225 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:37.773202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f1fe1-b58d-4ca1-b979-bc4d2836d225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:37.815402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2f3f1fe1-b58d-4ca1-b979-bc4d2836d225 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:37.817148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2f3f1fe1-b58d-4ca1-b979-bc4d2836d225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:37.906105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1073/2153] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:37.915144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea381aa-0b33-4d47-9910-fd6a7521bb94 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:37.915789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea381aa-0b33-4d47-9910-fd6a7521bb94 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:37.916208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea381aa-0b33-4d47-9910-fd6a7521bb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:37.949287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bea381aa-0b33-4d47-9910-fd6a7521bb94 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:37.962750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bea381aa-0b33-4d47-9910-fd6a7521bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:37.989522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1081/2154] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:37.990928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f1fe1-b58d-4ca1-b979-bc4d2836d225 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.010799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f1fe1-b58d-4ca1-b979-bc4d2836d225 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.024768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.025728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.027506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.028600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.031082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:38.033505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.039742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.068350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1074/2155] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:34:38.072991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea381aa-0b33-4d47-9910-fd6a7521bb94 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.086365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea381aa-0b33-4d47-9910-fd6a7521bb94 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.092493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.101482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.101482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.101482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.101482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.101482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.193446 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1082/2156] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:38 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:38.200362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.213238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42342a7e-f8ef-4616-ad5d-a708bfc7a048 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.279851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.280511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.280854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.281199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.281912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.284177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.378090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1075/2157] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:38 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:38.381970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.394603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76bb8e-42ee-414c-af00-79d0b9c5116c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.399542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3f3d2-9c87-4b76-89a6-f1ae61d9ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.400156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3f3d2-9c87-4b76-89a6-f1ae61d9ba3c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.400438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3f3d2-9c87-4b76-89a6-f1ae61d9ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.400710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3f3d2-9c87-4b76-89a6-f1ae61d9ba3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.406378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3f3d2-9c87-4b76-89a6-f1ae61d9ba3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.406378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3f3d2-9c87-4b76-89a6-f1ae61d9ba3c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.426405 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1083/2158] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:38.431158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.441800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533094e-d73d-4673-9131-2ec40c0c9d0c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.447973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54140c72-2231-42ee-9454-f97ea9425307 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.448466 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54140c72-2231-42ee-9454-f97ea9425307 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.449418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54140c72-2231-42ee-9454-f97ea9425307 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.451275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1084/2159] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:34:38.455792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7f25-c267-46cc-89ea-b3f3b283b80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.457208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7f25-c267-46cc-89ea-b3f3b283b80b admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.457725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7f25-c267-46cc-89ea-b3f3b283b80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.458147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7f25-c267-46cc-89ea-b3f3b283b80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.460010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7f25-c267-46cc-89ea-b3f3b283b80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.461138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7f25-c267-46cc-89ea-b3f3b283b80b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.475043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1076/2160] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:38.482799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54140c72-2231-42ee-9454-f97ea9425307 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.499304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.500155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.500512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.501554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54140c72-2231-42ee-9454-f97ea9425307 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.506944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f86ce3-1130-46b2-b6e4-181b99fffda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.507902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f86ce3-1130-46b2-b6e4-181b99fffda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.508334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f86ce3-1130-46b2-b6e4-181b99fffda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.508756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f86ce3-1130-46b2-b6e4-181b99fffda2 admin admin] PATH_INFO: `/v3/projects/6adff57d14564e5089786cfc2e21de71/users/817f65ead1354914865adc350dceef8b/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.526571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:38.530973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:38.545629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f86ce3-1130-46b2-b6e4-181b99fffda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6adff57d14564e5089786cfc2e21de71, user_id=817f65ead1354914865adc350dceef8b, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.549113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f86ce3-1130-46b2-b6e4-181b99fffda2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.583385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:38.629090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1085/2161] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:38 2022] PUT /identity/v3/projects/6adff57d14564e5089786cfc2e21de71/users/817f65ead1354914865adc350dceef8b/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:38.640515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.640864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.641221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.669345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1077/2162] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:34:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:38.686572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] MFA Rules not processed for user `817f65ead1354914865adc350dceef8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:38.688192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:38.739108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.740623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:38.763693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb872be-08bc-497a-98e8-dcfc3438cb87 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.812803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1086/2163] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:38.829836 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:38.831497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:38.899445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:38.930660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c891d2-1df4-4b08-b6d5-5407660427c3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:38.937418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.938065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.938346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.938788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.947862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:38.948552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.950109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:38.950584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:38.951113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.951969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:38.961692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1087/2164] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:38.963400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:38.965875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:38.978431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1078/2165] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:39.916974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:39.919698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:40.045248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e9e3f-ab54-488f-b9a8-e1ac499fdcd5 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:40.050113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:40.050848 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:40.051205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:40.051571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] PATH_INFO: `/v3/users/ebb70db8cca44c678082741c62e3ef44` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:40.057795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb70db8cca44c678082741c62e3ef44)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:40.060471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:40.941190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb70db8cca44c678082741c62e3ef44'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.086306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb70db8cca44c678082741c62e3ef44'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.106620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb70db8cca44c678082741c62e3ef44'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.107827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb70db8cca44c678082741c62e3ef44'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.109214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb70db8cca44c678082741c62e3ef44'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.128093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1088/2166] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:39 2022] DELETE /identity/v3/users/ebb70db8cca44c678082741c62e3ef44 => generated 0 bytes in 1209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:41.534171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:41.571929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d74f2a-ac24-426b-a573-1bf7028479b7 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:41.660681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:41.661341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:41.661626 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:41.661902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] PATH_INFO: `/v3/projects/99edc55b227143f8a539d293d72bf110` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:41.663140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=99edc55b227143f8a539d293d72bf110)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:41.665599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:41.837782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:41.840544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:41.850774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99edc55b227143f8a539d293d72bf110', 'request_id': 'req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.855593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:41.861701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99edc55b227143f8a539d293d72bf110', 'request_id': 'req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.979411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485f316-7e15-44b2-b8f6-c3b6be2fee4c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:41.992728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99edc55b227143f8a539d293d72bf110', 'request_id': 'req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:41.999950 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:42.008047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] The token cache is being invalidate because project 99edc55b227143f8a539d293d72bf110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:42.008362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:42.009126 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:42.013686 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1079/2167] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:41 2022] DELETE /identity/v3/projects/99edc55b227143f8a539d293d72bf110 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:42.014946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:42.016405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:42.054470 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:42.055130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:42.055415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:42.056887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:42.059273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:42.127559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1808e9-51c8-470b-8cd3-5c2bde00f8af admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:42.135774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:42.140792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:42.140792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:42.140792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] PATH_INFO: `/v3/users/4e85bc60be8840f0ab33734649ba8019` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:42.140792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e85bc60be8840f0ab33734649ba8019)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:42.140792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:42.161425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:42.164359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:42.181899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1089/2168] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:34:42.624898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e85bc60be8840f0ab33734649ba8019'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:42.735522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e85bc60be8840f0ab33734649ba8019'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:42.748268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e85bc60be8840f0ab33734649ba8019'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:42.755386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e85bc60be8840f0ab33734649ba8019'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:42.756097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-94903a59-ce79-43f5-a208-088984f47a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e85bc60be8840f0ab33734649ba8019'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:42.762477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1080/2169] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:42 2022] DELETE /identity/v3/users/4e85bc60be8840f0ab33734649ba8019 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:43.014235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:43.026300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e457234-a680-4754-8452-be9e451f6897 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:43.182098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:43.183616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:43.184234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:43.185073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] PATH_INFO: `/v3/projects/e5650dee65484f59884ef3b1aa90e934` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:43.186434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5650dee65484f59884ef3b1aa90e934)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:43.188638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:43.298912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5650dee65484f59884ef3b1aa90e934', 'request_id': 'req-78261a0e-0d96-4009-941b-0bc860cabfc0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:43.377544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5650dee65484f59884ef3b1aa90e934', 'request_id': 'req-78261a0e-0d96-4009-941b-0bc860cabfc0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:43.385140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5650dee65484f59884ef3b1aa90e934', 'request_id': 'req-78261a0e-0d96-4009-941b-0bc860cabfc0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:43.406027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] The token cache is being invalidate because project e5650dee65484f59884ef3b1aa90e934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:43.406133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:43.406838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-78261a0e-0d96-4009-941b-0bc860cabfc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:43.410627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1090/2170] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:43 2022] DELETE /identity/v3/projects/e5650dee65484f59884ef3b1aa90e934 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:43.413242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cace2e7-07f4-4622-9610-90c31f407d80 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:43.413529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cace2e7-07f4-4622-9610-90c31f407d80 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:43.413805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cace2e7-07f4-4622-9610-90c31f407d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:43.450759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4cace2e7-07f4-4622-9610-90c31f407d80 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:43.452043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4cace2e7-07f4-4622-9610-90c31f407d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:43.532441 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:43.560533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1081/2171] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:43.563452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971747-b316-42bc-bacd-6670527c611b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:43.563739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971747-b316-42bc-bacd-6670527c611b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:43.564036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce971747-b316-42bc-bacd-6670527c611b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:43.594775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ce971747-b316-42bc-bacd-6670527c611b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:43.598189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ce971747-b316-42bc-bacd-6670527c611b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:43.614682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1091/2172] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:43.626395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cace2e7-07f4-4622-9610-90c31f407d80 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:43.637160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cace2e7-07f4-4622-9610-90c31f407d80 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:43.642113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:43.644045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:43.644313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:43.644587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:43.645262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:43.645542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:43.647338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:43.661678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1082/2173] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:43.666164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce971747-b316-42bc-bacd-6670527c611b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:43.676420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce971747-b316-42bc-bacd-6670527c611b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:43.689637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:43.691965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:43.692392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:43.692667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:43.693310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:43.695275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:43.794816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1092/2174] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:43 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:43.799098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:43.810972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3b0f3-2d51-46c6-9d97-f3cf818616ec admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:43.882851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:43.883556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:43.883893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:43.884227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:43.884952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:43.888228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:43.967677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1083/2175] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:43 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:43.971842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:43.972984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:43.984270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a0c27-ffdb-4ec1-9715-f074b3ef0b3d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:43.998482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:43.999124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:43.999403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:43.999673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.000467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:44.002160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:44.015494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1093/2176] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:44.019292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:44.031356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93025763-e132-4b0e-acfe-6da10d736b78 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:44.037453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67f6ed-1de2-4264-95b1-5d57ddc68bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:44.038071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f6ed-1de2-4264-95b1-5d57ddc68bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:44.038481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f6ed-1de2-4264-95b1-5d57ddc68bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:44.038831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67f6ed-1de2-4264-95b1-5d57ddc68bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.039549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f6ed-1de2-4264-95b1-5d57ddc68bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:44.042327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67f6ed-1de2-4264-95b1-5d57ddc68bb0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:44.056976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1084/2177] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:34:44.061284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:44.072314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20c8db-6218-4a5b-bd1b-58bd54b841e5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:44.079059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:44.079722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:44.079999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:44.080266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] PATH_INFO: `/v3/projects/5aeab470de094079bc7880b29599327c/users/1627c90c84284a0885aef5c2a7c6c9e2/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.101781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aeab470de094079bc7880b29599327c, user_id=1627c90c84284a0885aef5c2a7c6c9e2, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:44.112162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:44.186593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1094/2178] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:44 2022] PUT /identity/v3/projects/5aeab470de094079bc7880b29599327c/users/1627c90c84284a0885aef5c2a7c6c9e2/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:44.197843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec969058-7cf6-4a8a-83ea-1ae3b2278f50 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:44.198125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec969058-7cf6-4a8a-83ea-1ae3b2278f50 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:44.199118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec969058-7cf6-4a8a-83ea-1ae3b2278f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.230339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ec969058-7cf6-4a8a-83ea-1ae3b2278f50 None None] MFA Rules not processed for user `1627c90c84284a0885aef5c2a7c6c9e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:44.231564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ec969058-7cf6-4a8a-83ea-1ae3b2278f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:44.299523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1085/2179] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:44.311275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:44.312932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:44.373387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d2d21-35de-489e-baa3-20c5839431bf admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:44.377058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077517-58e0-43dc-a8a0-af2a04d4bf07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:44.377651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077517-58e0-43dc-a8a0-af2a04d4bf07 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:44.377926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077517-58e0-43dc-a8a0-af2a04d4bf07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:44.378190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077517-58e0-43dc-a8a0-af2a04d4bf07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.387437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e077517-58e0-43dc-a8a0-af2a04d4bf07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:44.388855 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e077517-58e0-43dc-a8a0-af2a04d4bf07 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:44.398185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1095/2180] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:44.420605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec969058-7cf6-4a8a-83ea-1ae3b2278f50 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:44.479465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:44.485014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec969058-7cf6-4a8a-83ea-1ae3b2278f50 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:44.489772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:44.490423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:44.490702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:44.490976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] PATH_INFO: `/v3/users/1627c90c84284a0885aef5c2a7c6c9e2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.492105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1627c90c84284a0885aef5c2a7c6c9e2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:44.494017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:44.725708 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1627c90c84284a0885aef5c2a7c6c9e2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:44.792106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1627c90c84284a0885aef5c2a7c6c9e2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:44.804616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1627c90c84284a0885aef5c2a7c6c9e2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:44.812669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1627c90c84284a0885aef5c2a7c6c9e2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:44.813350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1627c90c84284a0885aef5c2a7c6c9e2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:44.819187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1086/2181] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:44 2022] DELETE /identity/v3/users/1627c90c84284a0885aef5c2a7c6c9e2 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:44.824502 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:44.824913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:44.825272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.844636 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:44.846046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:44.904658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1096/2182] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:44.914224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:44.915678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:44.972082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa29722-9631-4d1d-b728-c14f1eba7109 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:44.975662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fac00-312d-4fd8-818f-9effc0d0d482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:44.976315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fac00-312d-4fd8-818f-9effc0d0d482 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:44.976533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fac00-312d-4fd8-818f-9effc0d0d482 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:44.976799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fac00-312d-4fd8-818f-9effc0d0d482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:44.986308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fac00-312d-4fd8-818f-9effc0d0d482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:44.987754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fac00-312d-4fd8-818f-9effc0d0d482 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:44.997813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1087/2183] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:45.424255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:45.425682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:45.439286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c8cb0-e8f9-4876-abfd-f9b7d60b6d05 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:45.444555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:45.445185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:45.445609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:45.445896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] PATH_INFO: `/v3/projects/5aeab470de094079bc7880b29599327c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:45.447224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aeab470de094079bc7880b29599327c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:45.449923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:45.546720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:45.556477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aeab470de094079bc7880b29599327c', 'request_id': 'req-f3cb7dc4-7fcb-47bd-8191-005073892fb3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:45.620992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aeab470de094079bc7880b29599327c', 'request_id': 'req-f3cb7dc4-7fcb-47bd-8191-005073892fb3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:45.627620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aeab470de094079bc7880b29599327c', 'request_id': 'req-f3cb7dc4-7fcb-47bd-8191-005073892fb3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:45.638422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] The token cache is being invalidate because project 5aeab470de094079bc7880b29599327c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:45.638705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:45.639322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3cb7dc4-7fcb-47bd-8191-005073892fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:45.640798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1097/2184] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:45 2022] DELETE /identity/v3/projects/5aeab470de094079bc7880b29599327c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:45.645660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34831de1-309a-4dbc-b379-d14cc70b2600 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:45.645938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34831de1-309a-4dbc-b379-d14cc70b2600 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:45.646204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34831de1-309a-4dbc-b379-d14cc70b2600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:45.664969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-34831de1-309a-4dbc-b379-d14cc70b2600 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:45.666168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-34831de1-309a-4dbc-b379-d14cc70b2600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:45.725208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1088/2185] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:45.733703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1da079-ad93-4b40-b643-5e782207134e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:45.734014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1da079-ad93-4b40-b643-5e782207134e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:45.734311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1da079-ad93-4b40-b643-5e782207134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:45.755688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8e1da079-ad93-4b40-b643-5e782207134e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:45.756877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e1da079-ad93-4b40-b643-5e782207134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:45.765408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1098/2186] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:45.769657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34831de1-309a-4dbc-b379-d14cc70b2600 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:45.779430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34831de1-309a-4dbc-b379-d14cc70b2600 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:45.784111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:45.784700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:45.784976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:45.785249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:45.785916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:34:45.786217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:45.787846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:45.800111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1089/2187] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:34:45.804507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1da079-ad93-4b40-b643-5e782207134e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:45.814767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1da079-ad93-4b40-b643-5e782207134e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:45.819385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:45.819974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:45.820250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:45.820518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:45.821138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:45.823070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:45.908432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1099/2188] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:45 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:45.912599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:45.919586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:45.925148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff239c-83f9-45af-a46d-fc0ff782ca90 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:45.982206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:45.982732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:45.983007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:45.983276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:45.983880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:45.985804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:46.073966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1090/2189] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:45 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:46.078219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:46.089566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f44801-5249-4162-8bd0-e3315fc3ae4a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:46.095736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:46.096337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:46.096612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:46.096883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:46.097559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:46.099192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:46.110666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1100/2190] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:46.114796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:46.125414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f82fb2-d0b6-4e73-9e65-50778fc700dc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:46.130209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe51195-10c4-4a53-84fe-143f69be9b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:46.130840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51195-10c4-4a53-84fe-143f69be9b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:46.136151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51195-10c4-4a53-84fe-143f69be9b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:46.136151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51195-10c4-4a53-84fe-143f69be9b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:46.136151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe51195-10c4-4a53-84fe-143f69be9b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:46.136151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe51195-10c4-4a53-84fe-143f69be9b57 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:46.146849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1091/2191] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:46.151194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:46.162468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d836f-cafc-4f85-b7f9-90a28a1a1f68 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:46.168007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:46.168785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:46.169109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:46.169386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] PATH_INFO: `/v3/projects/f99e0384406446c2a7a0c880faf8aef4/users/e5af67ad52514da4846e4ef8c53f9781/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:46.185148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99e0384406446c2a7a0c880faf8aef4, user_id=e5af67ad52514da4846e4ef8c53f9781, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:46.187901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:46.256512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1101/2192] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:46 2022] PUT /identity/v3/projects/f99e0384406446c2a7a0c880faf8aef4/users/e5af67ad52514da4846e4ef8c53f9781/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:46.267213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:46.267516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:46.267786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:46.298245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] MFA Rules not processed for user `e5af67ad52514da4846e4ef8c53f9781`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:46.299461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:46.363369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1092/2193] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:34:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:46.374642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:46.376349 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:46.433643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57c91-4f40-4596-b61a-3b16efd88e88 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:46.437325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f4d7-ebab-4fdc-bd7e-99a3ccd04daa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:46.437924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f4d7-ebab-4fdc-bd7e-99a3ccd04daa service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:46.438201 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f4d7-ebab-4fdc-bd7e-99a3ccd04daa service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:46.438625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f4d7-ebab-4fdc-bd7e-99a3ccd04daa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:46.447287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f4d7-ebab-4fdc-bd7e-99a3ccd04daa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:46.449227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f4d7-ebab-4fdc-bd7e-99a3ccd04daa service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:46.458298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1102/2194] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:56.360414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:56.454129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:59.787224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:59.788681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:59.859433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8119b-9d36-4728-b81b-da479e4a9e91 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:59.864496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:59.865171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:59.865454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:59.865726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] PATH_INFO: `/v3/users/08ff23d43abb4c128a3c00a9654d552d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:59.866923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08ff23d43abb4c128a3c00a9654d552d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:59.868953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:00.071568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08ff23d43abb4c128a3c00a9654d552d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.153375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08ff23d43abb4c128a3c00a9654d552d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.166245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08ff23d43abb4c128a3c00a9654d552d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.174212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08ff23d43abb4c128a3c00a9654d552d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.174948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08ff23d43abb4c128a3c00a9654d552d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.180710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1093/2195] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:59 2022] DELETE /identity/v3/users/08ff23d43abb4c128a3c00a9654d552d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:00.185805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:00.186319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:00.186319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:00.206215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:00.207481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.266977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1103/2196] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:00.276687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:00.278668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.334861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd6b7c-b808-4fa7-af2c-2a20b6835c37 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:00.338834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2416f5e-31a8-4173-ba58-b2f7ab682bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:00.339429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2416f5e-31a8-4173-ba58-b2f7ab682bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:00.339709 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2416f5e-31a8-4173-ba58-b2f7ab682bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:00.339982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2416f5e-31a8-4173-ba58-b2f7ab682bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:00.349913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2416f5e-31a8-4173-ba58-b2f7ab682bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:00.351388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2416f5e-31a8-4173-ba58-b2f7ab682bd7 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:00.362180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1094/2197] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:00.416896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:00.418321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.431469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cceb5b-5aec-4539-a157-96a64ed1c416 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:00.436376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:00.437282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:00.437282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:00.437540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] PATH_INFO: `/v3/users/e5af67ad52514da4846e4ef8c53f9781` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:00.438734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5af67ad52514da4846e4ef8c53f9781)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:00.440645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:00.620720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5af67ad52514da4846e4ef8c53f9781'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.731422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5af67ad52514da4846e4ef8c53f9781'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.744715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5af67ad52514da4846e4ef8c53f9781'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.745399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5af67ad52514da4846e4ef8c53f9781'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.746014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5af67ad52514da4846e4ef8c53f9781'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:00.759707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1104/2198] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:00 2022] DELETE /identity/v3/users/e5af67ad52514da4846e4ef8c53f9781 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:00.765359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:00.765774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:00.766068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:00.786611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:00.787849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.815070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:00.815070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.827054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:00.877198 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1095/2199] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:00.878823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fcaf3-b8cd-4bf8-a77f-16b046e46abd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:00.883979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:00.884823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:00.884937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:00.885204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] PATH_INFO: `/v3/projects/e431f63014204603ab0dba1d4fe646f4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:00.890661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e431f63014204603ab0dba1d4fe646f4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:00.890801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:00.890801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.890973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:00.913781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:00.971416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577f28c-ef41-436a-937e-706cab765ed8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:00.977087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:00.977917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:00.978401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:00.978846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:00.991165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:00.992729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:01.011124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1096/2200] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:35:01.021945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e431f63014204603ab0dba1d4fe646f4', 'request_id': 'req-5632e573-533c-41ab-a511-4fe3d24c53b5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.119372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e431f63014204603ab0dba1d4fe646f4', 'request_id': 'req-5632e573-533c-41ab-a511-4fe3d24c53b5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.127746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e431f63014204603ab0dba1d4fe646f4', 'request_id': 'req-5632e573-533c-41ab-a511-4fe3d24c53b5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.139320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] The token cache is being invalidate because project e431f63014204603ab0dba1d4fe646f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:01.139612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.140286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.141831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1105/2201] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:00 2022] DELETE /identity/v3/projects/e431f63014204603ab0dba1d4fe646f4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:01.146584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:01.158201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8acfc-44ef-49fa-90ce-a89c8a451601 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:01.228931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:01.229560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:01.229836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:01.230102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] PATH_INFO: `/v3/users/d1c7a0e1c87a4f1591ad9d22e8266d72` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:01.231769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c7a0e1c87a4f1591ad9d22e8266d72)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:01.233849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:01.459084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c7a0e1c87a4f1591ad9d22e8266d72'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.559854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c7a0e1c87a4f1591ad9d22e8266d72'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.570582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:01.572913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c7a0e1c87a4f1591ad9d22e8266d72'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.579701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c7a0e1c87a4f1591ad9d22e8266d72'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.580603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c7a0e1c87a4f1591ad9d22e8266d72'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.587719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1097/2202] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:01 2022] DELETE /identity/v3/users/d1c7a0e1c87a4f1591ad9d22e8266d72 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:01.607292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:01.609490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:01.642201 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632e573-533c-41ab-a511-4fe3d24c53b5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:01.745812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:01.746981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:01.747467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:01.747955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] PATH_INFO: `/v3/projects/f99e0384406446c2a7a0c880faf8aef4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:01.750062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99e0384406446c2a7a0c880faf8aef4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:01.753910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:01.842675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:01.844090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:01.859216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb56fd-7ad0-4a79-9f68-5df901907735 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:01.864297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:01.865109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:01.865387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:01.865664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] PATH_INFO: `/v3/users/817f65ead1354914865adc350dceef8b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:01.866984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=817f65ead1354914865adc350dceef8b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:01.869227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:01.876771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99e0384406446c2a7a0c880faf8aef4', 'request_id': 'req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.959588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99e0384406446c2a7a0c880faf8aef4', 'request_id': 'req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.969742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99e0384406446c2a7a0c880faf8aef4', 'request_id': 'req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.990339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] The token cache is being invalidate because project f99e0384406446c2a7a0c880faf8aef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:01.990720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.991647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b01ea0f1-42a0-4354-a4f4-d50dc45b9708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:01.993899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1106/2203] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:01 2022] DELETE /identity/v3/projects/f99e0384406446c2a7a0c880faf8aef4 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:01.999390 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e92d2d-2404-43d4-b047-60d2dbbf896e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:01.999803 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e92d2d-2404-43d4-b047-60d2dbbf896e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.000553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e92d2d-2404-43d4-b047-60d2dbbf896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.028205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a6e92d2d-2404-43d4-b047-60d2dbbf896e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:02.032648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a6e92d2d-2404-43d4-b047-60d2dbbf896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.063612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:02.084542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '817f65ead1354914865adc350dceef8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.097416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1107/2204] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:02.108024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.108311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.108600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.146484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:02.148454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.154598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '817f65ead1354914865adc350dceef8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.160549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1108/2205] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:35:02.160549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.166498 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.193520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd501bd-ad84-4ef2-bdbe-06dbce79926e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.194114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '817f65ead1354914865adc350dceef8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.203308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.205534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.205997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.206478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] PATH_INFO: `/v3/projects/5b94911b4ab94a26a28f37cc8fdf9ea0` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.207933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b94911b4ab94a26a28f37cc8fdf9ea0)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.211134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.216028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '817f65ead1354914865adc350dceef8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.217333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '817f65ead1354914865adc350dceef8b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.249756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1098/2206] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:01 2022] DELETE /identity/v3/users/817f65ead1354914865adc350dceef8b => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:02.250898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.253976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.327394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d02e09-de3e-4462-a3f4-0cd490b593f4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.351027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b94911b4ab94a26a28f37cc8fdf9ea0', 'request_id': 'req-83ced6ef-076d-4880-806a-260632202273'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.414006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cd0a2-9050-4b47-9a6a-acbad3b96ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.414699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cd0a2-9050-4b47-9a6a-acbad3b96ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.415120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cd0a2-9050-4b47-9a6a-acbad3b96ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.415367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cd0a2-9050-4b47-9a6a-acbad3b96ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.415932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cd0a2-9050-4b47-9a6a-acbad3b96ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:02.416234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cd0a2-9050-4b47-9a6a-acbad3b96ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.417862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cd0a2-9050-4b47-9a6a-acbad3b96ebf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.425204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:02.430341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1099/2207] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:02.435000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8511df-5ed6-4f11-b7b9-a9d43b4b0c36 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.435000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8511df-5ed6-4f11-b7b9-a9d43b4b0c36 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.435000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8511df-5ed6-4f11-b7b9-a9d43b4b0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.452936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8f8511df-5ed6-4f11-b7b9-a9d43b4b0c36 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:02.453852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8f8511df-5ed6-4f11-b7b9-a9d43b4b0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.472000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1100/2208] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:02.472995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8511df-5ed6-4f11-b7b9-a9d43b4b0c36 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.486375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8511df-5ed6-4f11-b7b9-a9d43b4b0c36 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.486518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b94911b4ab94a26a28f37cc8fdf9ea0', 'request_id': 'req-83ced6ef-076d-4880-806a-260632202273'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.491051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.492360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.492738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.493194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.493968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.495949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b94911b4ab94a26a28f37cc8fdf9ea0', 'request_id': 'req-83ced6ef-076d-4880-806a-260632202273'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.496358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.512519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] The token cache is being invalidate because project 5b94911b4ab94a26a28f37cc8fdf9ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:02.512866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.513564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:02.515939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1109/2209] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:02 2022] DELETE /identity/v3/projects/5b94911b4ab94a26a28f37cc8fdf9ea0 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:02.516742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.519609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.591761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ced6ef-076d-4880-806a-260632202273 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.597483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.598546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.599188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.599637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.602959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.605861 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1101/2210] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:02 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:02.607482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.609867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.617156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.619304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.631144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1110/2211] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:02.632109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.633864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:02.647399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62dcbf-762f-439f-a285-7349d068f7fe service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.702066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537eda0-0715-40c9-8e99-ac98553c2186 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.711206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.711860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.712136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.712406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] PATH_INFO: `/v3/users/6c4120da1223409ebcc9954768d35180` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.712931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.714011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.714334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4120da1223409ebcc9954768d35180)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.714476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.714905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.716219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.717579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.718655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.817061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1111/2212] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:02 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:02.821016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.831484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723a025-4209-46ea-acc6-0120b4a2793d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.836038 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.837618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.837618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.837618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.837879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.839440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.849809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1112/2213] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:02.853573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.860809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:02.866241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f650f27-0135-4b2f-be8c-42c06fc395c7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.870892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.873503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.873503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.873503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.873503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.874575 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.890385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1113/2214] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:02.892150 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:02.905415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d65a68-6b48-458d-8bc6-58a97117fe6f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:02.912103 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b873360-8c32-41c2-bd8f-879c841eb2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:02.913672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b873360-8c32-41c2-bd8f-879c841eb2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:02.913837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b873360-8c32-41c2-bd8f-879c841eb2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:02.914112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b873360-8c32-41c2-bd8f-879c841eb2ec admin admin] PATH_INFO: `/v3/projects/5ec7a76094c54d6c8e2517e254f44c16/users/f67323f672464220a38906660715a9d6/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:02.934344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b873360-8c32-41c2-bd8f-879c841eb2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec7a76094c54d6c8e2517e254f44c16, user_id=f67323f672464220a38906660715a9d6, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:02.937493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b873360-8c32-41c2-bd8f-879c841eb2ec admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:02.974576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4120da1223409ebcc9954768d35180'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.018491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1114/2215] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:02 2022] PUT /identity/v3/projects/5ec7a76094c54d6c8e2517e254f44c16/users/f67323f672464220a38906660715a9d6/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:03.022616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.022980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.023345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.040605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4120da1223409ebcc9954768d35180'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.058933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4120da1223409ebcc9954768d35180'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.061143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:03.062558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:03.070136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4120da1223409ebcc9954768d35180'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.070952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4120da1223409ebcc9954768d35180'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.079561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1102/2216] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:02 2022] DELETE /identity/v3/users/6c4120da1223409ebcc9954768d35180 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:03.125619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1115/2217] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:35:03.139146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:03.139146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:03.170116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:03.171618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:03.185973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c3aa5-9958-4f14-85a6-60932767120f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:03.202388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:03.202388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffcb2f-f75a-4fac-962b-f454b7bf9def admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:03.204518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e015b-d6d5-4fdc-8ad4-ee5e88f4a011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:03.205449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e015b-d6d5-4fdc-8ad4-ee5e88f4a011 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.206230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e015b-d6d5-4fdc-8ad4-ee5e88f4a011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.206464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e015b-d6d5-4fdc-8ad4-ee5e88f4a011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.266171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:03.266930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.267280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.267626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] PATH_INFO: `/v3/projects/6adff57d14564e5089786cfc2e21de71` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.268847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6adff57d14564e5089786cfc2e21de71)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:03.270875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:03.283975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e015b-d6d5-4fdc-8ad4-ee5e88f4a011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:03.285405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e015b-d6d5-4fdc-8ad4-ee5e88f4a011 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:03.294899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1103/2218] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:03.438430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6adff57d14564e5089786cfc2e21de71', 'request_id': 'req-b5e64149-ecb8-4e2a-9b9a-ac088413e226'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.530787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6adff57d14564e5089786cfc2e21de71', 'request_id': 'req-b5e64149-ecb8-4e2a-9b9a-ac088413e226'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.538323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6adff57d14564e5089786cfc2e21de71', 'request_id': 'req-b5e64149-ecb8-4e2a-9b9a-ac088413e226'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.550376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] The token cache is being invalidate because project 6adff57d14564e5089786cfc2e21de71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:03.550925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.553053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5e64149-ecb8-4e2a-9b9a-ac088413e226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.555023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1116/2219] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:03 2022] DELETE /identity/v3/projects/6adff57d14564e5089786cfc2e21de71 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:03.561336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.562091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.563362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.593920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:03.596119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:03.665733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1104/2220] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:03.670557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb939ed-2a53-41c1-a41f-65f1162bc877 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.670982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb939ed-2a53-41c1-a41f-65f1162bc877 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.671600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb939ed-2a53-41c1-a41f-65f1162bc877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.694971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8fb939ed-2a53-41c1-a41f-65f1162bc877 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:03.696205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8fb939ed-2a53-41c1-a41f-65f1162bc877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:03.710135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1117/2221] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:03.711381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:03.712084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb939ed-2a53-41c1-a41f-65f1162bc877 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:03.713058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:03.722804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb939ed-2a53-41c1-a41f-65f1162bc877 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:03.730391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403faf9d-0cb6-41d1-9b57-9a7c5bac5f18 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:03.735612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:03.736166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.736397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.737133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.737363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:03.737659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:03.739859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:03.741496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:03.742614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.743555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.744173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] PATH_INFO: `/v3/projects/f4f13e6bb10f4b12b10d68091d18f8aa` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.745835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f13e6bb10f4b12b10d68091d18f8aa)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:03.749329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:03.756842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1118/2222] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:03.770872 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:03.773327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f20c2-dd39-4331-8f5c-3c04288c367e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:03.780204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:03.780934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.781358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.781853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:03.782623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:03.784630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:03.881527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f13e6bb10f4b12b10d68091d18f8aa', 'request_id': 'req-a0e0cb43-c5d0-4403-9a24-7f990d307026'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.892435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f13e6bb10f4b12b10d68091d18f8aa', 'request_id': 'req-a0e0cb43-c5d0-4403-9a24-7f990d307026'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.951344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1119/2223] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:03 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:35:03.952443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:03.969471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f13e6bb10f4b12b10d68091d18f8aa', 'request_id': 'req-a0e0cb43-c5d0-4403-9a24-7f990d307026'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.975148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:03.982929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] The token cache is being invalidate because project f4f13e6bb10f4b12b10d68091d18f8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:03.983454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.984142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a0e0cb43-c5d0-4403-9a24-7f990d307026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:03.986748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1105/2224] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:03 2022] DELETE /identity/v3/projects/f4f13e6bb10f4b12b10d68091d18f8aa => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:03.992860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db934a-12ca-40ad-808c-c8cc3fdbe0c2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:03.993245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db934a-12ca-40ad-808c-c8cc3fdbe0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:03.993619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db934a-12ca-40ad-808c-c8cc3fdbe0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.021733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-69db934a-12ca-40ad-808c-c8cc3fdbe0c2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:04.023735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-69db934a-12ca-40ad-808c-c8cc3fdbe0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:04.036470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1106/2225] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:04.038723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae3f74-9e06-457e-bc96-be69716e336b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.045054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762c48d-a64c-42e8-ab63-553206dd7cca None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.045789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762c48d-a64c-42e8-ab63-553206dd7cca None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.045789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762c48d-a64c-42e8-ab63-553206dd7cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.047450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.048335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.048825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.049338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.050177 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.052394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.098327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7762c48d-a64c-42e8-ab63-553206dd7cca None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:04.100552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7762c48d-a64c-42e8-ab63-553206dd7cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:04.111337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1107/2226] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:04.115265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762c48d-a64c-42e8-ab63-553206dd7cca None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.134273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762c48d-a64c-42e8-ab63-553206dd7cca None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.143372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1120/2227] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:03 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:04.143774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.144781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.145551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.145973 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.147257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:04.147719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.147842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.149752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.150885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:04.172767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a800c-f5cc-40f2-b927-1b83a836c8ff admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.177456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1108/2228] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:35:04.183264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.192288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.192592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.192690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.192690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.192690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.192690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.206829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1121/2229] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:04.207580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b844638-56dd-4371-9c77-122578a9ec3a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.207977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.215999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.215999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.215999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.215999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.217120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.219086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.219248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c297cc-5cd5-4a94-a0e3-b85ab7195d4d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.224360 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.225085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.225745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.225745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.226558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.228362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.241375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1122/2230] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:04.245504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.257132 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1541e40-9d40-489d-9135-2efecde2f10f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.262899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520ce5a-ad9e-49a2-b2d2-5e9eadf20f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.264109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ce5a-ad9e-49a2-b2d2-5e9eadf20f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.269876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ce5a-ad9e-49a2-b2d2-5e9eadf20f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.269876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ce5a-ad9e-49a2-b2d2-5e9eadf20f7a admin admin] PATH_INFO: `/v3/projects/bde5689c6a904f7baa55df75664a8994/users/c4538f8ee0da4c45aa1245b0e4a97b94/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.281749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520ce5a-ad9e-49a2-b2d2-5e9eadf20f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde5689c6a904f7baa55df75664a8994, user_id=c4538f8ee0da4c45aa1245b0e4a97b94, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.284619 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520ce5a-ad9e-49a2-b2d2-5e9eadf20f7a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.326042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1109/2231] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:04 2022] POST /identity/v3/projects => generated 393 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:04.330702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.376962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1123/2232] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:04 2022] PUT /identity/v3/projects/bde5689c6a904f7baa55df75664a8994/users/c4538f8ee0da4c45aa1245b0e4a97b94/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:04.388121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.388605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.389051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.395381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:04.431157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] MFA Rules not processed for user `c4538f8ee0da4c45aa1245b0e4a97b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:04.433204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:04.450755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7514395-3e90-42e3-8db6-12777dfd5317 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.536808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1124/2233] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:04.542293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.542293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:04.588696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.589065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.589341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.589615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.591156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.594492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.632042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecca7a9-15fc-448f-abbe-0e4156ab2cdf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.638461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.640497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.640497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.640497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] PATH_INFO: `/v3/users/aee833b6e2444bf5a298ad45d2afeea8` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.641577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee833b6e2444bf5a298ad45d2afeea8)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.644034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.726737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1110/2234] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:04 2022] POST /identity/v3/users => generated 297 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:35:04.730026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.730026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:04.795240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4d2a-d10e-4b14-a374-f63d84b0bff9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.800050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.800942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.801758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.802136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.813720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.815369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.826155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1111/2235] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:04.827161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.864376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee833b6e2444bf5a298ad45d2afeea8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:04.866089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9f15d-d896-4f5c-b993-98356049c920 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.872804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.874122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.874645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.874985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.875700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.877542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:04.892425 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1112/2236] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:04.893497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:04.909166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ca412-3f27-4804-b91f-6104cfe4e51e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:04.916294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:04.916941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:04.917216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:04.921426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] PATH_INFO: `/v3/users/c4538f8ee0da4c45aa1245b0e4a97b94` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:04.921426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4538f8ee0da4c45aa1245b0e4a97b94)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:04.921999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:05.009530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee833b6e2444bf5a298ad45d2afeea8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.034482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee833b6e2444bf5a298ad45d2afeea8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.035906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee833b6e2444bf5a298ad45d2afeea8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.036745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee833b6e2444bf5a298ad45d2afeea8'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.051793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1125/2237] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:04 2022] DELETE /identity/v3/users/aee833b6e2444bf5a298ad45d2afeea8 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:05.052407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:05.053982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.129047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b86216-d60a-496b-a4d7-b53c04bc7b7d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:05.129436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4538f8ee0da4c45aa1245b0e4a97b94'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.134978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31688a9e-e3ff-442b-8717-2d3c43f0ce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:05.135608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31688a9e-e3ff-442b-8717-2d3c43f0ce36 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.135886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31688a9e-e3ff-442b-8717-2d3c43f0ce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.136152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31688a9e-e3ff-442b-8717-2d3c43f0ce36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.136792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31688a9e-e3ff-442b-8717-2d3c43f0ce36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:05.139238 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31688a9e-e3ff-442b-8717-2d3c43f0ce36 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:05.149415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1126/2238] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:05.150942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f9720-6290-4f6c-8ff3-68a89c2124f9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.151231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f9720-6290-4f6c-8ff3-68a89c2124f9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.151503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f9720-6290-4f6c-8ff3-68a89c2124f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.182870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9a9f9720-6290-4f6c-8ff3-68a89c2124f9 None None] MFA Rules not processed for user `f67323f672464220a38906660715a9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:05.184702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9a9f9720-6290-4f6c-8ff3-68a89c2124f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.227783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4538f8ee0da4c45aa1245b0e4a97b94'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.242363 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4538f8ee0da4c45aa1245b0e4a97b94'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.249830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4538f8ee0da4c45aa1245b0e4a97b94'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.250564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c7671fb0-a815-4f2f-9236-507b62fe4e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4538f8ee0da4c45aa1245b0e4a97b94'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:05.257021 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:05.257265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1113/2239] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:04 2022] DELETE /identity/v3/users/c4538f8ee0da4c45aa1245b0e4a97b94 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:05.258753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cc1c2-b4b1-4d42-9120-f04f7e0719ba None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.259136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cc1c2-b4b1-4d42-9120-f04f7e0719ba None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.259413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cc1c2-b4b1-4d42-9120-f04f7e0719ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.260490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1127/2240] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:05.265338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f9720-6290-4f6c-8ff3-68a89c2124f9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:05.277574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f9720-6290-4f6c-8ff3-68a89c2124f9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:05.280801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-447cc1c2-b4b1-4d42-9120-f04f7e0719ba None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:05.282187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-447cc1c2-b4b1-4d42-9120-f04f7e0719ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.345805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:05.346759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.347113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.347494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] PATH_INFO: `/v3/projects/79b38ce464a043f8a61f9535c7aa18de/users/bde4ce181f4942d58e01075e8fb59d60/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.362820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1114/2241] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:05.364139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.364457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.364754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.369682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b38ce464a043f8a61f9535c7aa18de, user_id=bde4ce181f4942d58e01075e8fb59d60, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:05.373108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:05.384585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:05.390622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.397777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1115/2242] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:05.397777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:05.399325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.456943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1128/2243] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:05 2022] PUT /identity/v3/projects/79b38ce464a043f8a61f9535c7aa18de/users/bde4ce181f4942d58e01075e8fb59d60/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:05.456943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:05.458980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.473054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b72fd5-94d1-44ca-a061-fc2f175d42d7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:05.530330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78c0f-b83d-4bfa-afb0-64a93521fd65 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:05.534927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:05.535727 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.536565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.536942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.540813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa55351-c714-4627-adfb-116b47b34003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:05.541430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa55351-c714-4627-adfb-116b47b34003 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.541712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa55351-c714-4627-adfb-116b47b34003 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.542327 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa55351-c714-4627-adfb-116b47b34003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.545436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eaa55351-c714-4627-adfb-116b47b34003 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.625633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:05.627223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:05.628343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa55351-c714-4627-adfb-116b47b34003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:05.629808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa55351-c714-4627-adfb-116b47b34003 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:05.638697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1129/2244] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:05.640055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:05.643434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1116/2245] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:05.644772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c15471-5fb7-491a-9a00-c5b7b80b65b1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.645171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c15471-5fb7-491a-9a00-c5b7b80b65b1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.645445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c15471-5fb7-491a-9a00-c5b7b80b65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.648917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabb6d5-e8e3-4ed0-8f5a-b78cac8b8d5b service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:05.654681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:05.657694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.657694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.657694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.677679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:05.679495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:05.690314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-17c15471-5fb7-491a-9a00-c5b7b80b65b1 None None] MFA Rules not processed for user `bde4ce181f4942d58e01075e8fb59d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:05.691870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-17c15471-5fb7-491a-9a00-c5b7b80b65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.705839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1130/2246] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:05.731455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:05.801838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1117/2247] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:05.802758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:05.805071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:05.843750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596696e-185e-431f-bbfc-6c777c312afa service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:05.861026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:05.861632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:05.861924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:05.863031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:05.863676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:05.865738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:06.029097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1131/2248] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:05 2022] POST /identity/v3/projects => generated 393 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:06.030486 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c15471-5fb7-491a-9a00-c5b7b80b65b1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:06.045553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c15471-5fb7-491a-9a00-c5b7b80b65b1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:06.113909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:06.114705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:06.115052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:06.115383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:06.116121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:06.118153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:06.220288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1118/2249] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:06 2022] POST /identity/v3/users => generated 303 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:06.225058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:06.285075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067d04b-d0e9-4e1b-b278-446b15c075a6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:06.292241 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:06.293258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:06.294649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:06.294921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:06.295932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:06.297671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:06.317004 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:06.322442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1132/2250] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:06.330744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc991e-9b30-41d6-a981-106241a3aab1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:06.339581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:06.340778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:06.341068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:06.341360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] PATH_INFO: `/v3/projects/cede2dbdc3e14aee9443dddcf4e10fa0/users/1d3fe869d5c74709b01e4d015109a463/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:06.370184 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cede2dbdc3e14aee9443dddcf4e10fa0, user_id=1d3fe869d5c74709b01e4d015109a463, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:06.373031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:06.427919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:06.430128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:06.463213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73635b56-38c1-495e-a783-ee7141a6a544 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:06.470281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:06.471083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:06.471369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:06.472775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] PATH_INFO: `/v3/projects/bbd654c229774aa8a91164c0b097bd26` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:06.494549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1119/2251] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:06 2022] PUT /identity/v3/projects/cede2dbdc3e14aee9443dddcf4e10fa0/users/1d3fe869d5c74709b01e4d015109a463/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:06.495406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:06.498608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:06.501616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd654c229774aa8a91164c0b097bd26)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:06.505458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:06.510330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:06.660642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:06.668376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e480ab-ded3-4b4e-a89b-71f8b8f791dd admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:06.676361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd654c229774aa8a91164c0b097bd26', 'request_id': 'req-e109e7e3-46c1-4528-b8f8-d947447a8129'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.743457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:06.743695 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:06.743915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:06.744165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] PATH_INFO: `/v3/projects/bde5689c6a904f7baa55df75664a8994` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:06.745961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde5689c6a904f7baa55df75664a8994)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:06.747480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:06.821603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd654c229774aa8a91164c0b097bd26', 'request_id': 'req-e109e7e3-46c1-4528-b8f8-d947447a8129'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.830707 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd654c229774aa8a91164c0b097bd26', 'request_id': 'req-e109e7e3-46c1-4528-b8f8-d947447a8129'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.848686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] The token cache is being invalidate because project bbd654c229774aa8a91164c0b097bd26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:06.849156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.850152 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.852283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1133/2252] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:06 2022] DELETE /identity/v3/projects/bbd654c229774aa8a91164c0b097bd26 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:06.853288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:06.865237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:06.877756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde5689c6a904f7baa55df75664a8994', 'request_id': 'req-5f0a6a21-240e-4269-86ac-e43b1bc581f8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.885854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde5689c6a904f7baa55df75664a8994', 'request_id': 'req-5f0a6a21-240e-4269-86ac-e43b1bc581f8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.904976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109e7e3-46c1-4528-b8f8-d947447a8129 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:06.974821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde5689c6a904f7baa55df75664a8994', 'request_id': 'req-5f0a6a21-240e-4269-86ac-e43b1bc581f8'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.990997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] The token cache is being invalidate because project bde5689c6a904f7baa55df75664a8994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:06.991479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.994986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5f0a6a21-240e-4269-86ac-e43b1bc581f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:06.997859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1120/2253] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:06 2022] DELETE /identity/v3/projects/bde5689c6a904f7baa55df75664a8994 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:07.001862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.002201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.006631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.006926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2906ba-bdea-49ea-9326-fc9a73461894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.007594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2906ba-bdea-49ea-9326-fc9a73461894 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.008013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2906ba-bdea-49ea-9326-fc9a73461894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.008326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2906ba-bdea-49ea-9326-fc9a73461894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.009034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2906ba-bdea-49ea-9326-fc9a73461894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.012709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2906ba-bdea-49ea-9326-fc9a73461894 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.029706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1134/2254] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:07.030524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c338fb3-144d-457e-9bf4-503ac00792e7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.030967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c338fb3-144d-457e-9bf4-503ac00792e7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.031544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c338fb3-144d-457e-9bf4-503ac00792e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.048482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:07.051576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.071053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1c338fb3-144d-457e-9bf4-503ac00792e7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:07.073042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c338fb3-144d-457e-9bf4-503ac00792e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.080260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1121/2255] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:07.081212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.083664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.085406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1135/2256] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:07.087512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b9a10-aeac-401b-93f0-eeeb3d1403ee None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.087907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b9a10-aeac-401b-93f0-eeeb3d1403ee None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.088325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b9a10-aeac-401b-93f0-eeeb3d1403ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.096534 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339338ba-7c2f-45aa-bb7d-693e4b1647e4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.102376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd83ab4-5ecc-480a-954b-c9dd981d58d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.104190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83ab4-5ecc-480a-954b-c9dd981d58d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.104190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83ab4-5ecc-480a-954b-c9dd981d58d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.104190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd83ab4-5ecc-480a-954b-c9dd981d58d3 admin admin] PATH_INFO: `/v3/projects/cede2dbdc3e14aee9443dddcf4e10fa0/users/1d3fe869d5c74709b01e4d015109a463/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.106508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd83ab4-5ecc-480a-954b-c9dd981d58d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cede2dbdc3e14aee9443dddcf4e10fa0, user_id=1d3fe869d5c74709b01e4d015109a463, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.109275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd83ab4-5ecc-480a-954b-c9dd981d58d3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.110782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-590b9a10-aeac-401b-93f0-eeeb3d1403ee None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:07.115456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-590b9a10-aeac-401b-93f0-eeeb3d1403ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.120480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1122/2257] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:07 2022] PUT /identity/v3/projects/cede2dbdc3e14aee9443dddcf4e10fa0/users/1d3fe869d5c74709b01e4d015109a463/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:07.129085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41da8ad-50ea-4214-b1a4-f817f954ebb2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.129646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41da8ad-50ea-4214-b1a4-f817f954ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.130116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41da8ad-50ea-4214-b1a4-f817f954ebb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.163819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a41da8ad-50ea-4214-b1a4-f817f954ebb2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:07.164480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a41da8ad-50ea-4214-b1a4-f817f954ebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.194082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1136/2258] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:07.195391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c614f-4c5e-48a7-b96b-07023d029a15 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.195677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c614f-4c5e-48a7-b96b-07023d029a15 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.195952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c614f-4c5e-48a7-b96b-07023d029a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.225769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-281c614f-4c5e-48a7-b96b-07023d029a15 None None] MFA Rules not processed for user `1d3fe869d5c74709b01e4d015109a463`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:07.227124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-281c614f-4c5e-48a7-b96b-07023d029a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.239816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1123/2259] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:07.251288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41da8ad-50ea-4214-b1a4-f817f954ebb2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.266759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41da8ad-50ea-4214-b1a4-f817f954ebb2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.279043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.279760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.280139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.280442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.281543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:07.281913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.284520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.300289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1124/2260] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:07.301324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.314195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e9f09-89cc-497c-bd3f-391ebfbc01ea admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.317239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1137/2261] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:07.317239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c614f-4c5e-48a7-b96b-07023d029a15 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.322976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.323649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.323978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.327408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.327408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:07.327408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.328169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.331429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c614f-4c5e-48a7-b96b-07023d029a15 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.338121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.338857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.339469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.339469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.342680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.343842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1125/2262] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:07.344855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.346882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.354557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.427013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4860bc8-2329-4a72-b6e2-a68407f1c37e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.431051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.431651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.433282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.433282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.442138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.443684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.454230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1126/2263] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:07.456209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.469488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73172-6bf3-4687-91b4-eab1ac969eac service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.475999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.476893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.477372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.477843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.478748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.481367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.538409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1138/2264] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/projects => generated 371 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:07.541880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.591106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1127/2265] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:07.592873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.698884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053c0d-7e16-406f-804b-f70a8e391515 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.730656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9268b-0e50-4fef-9b6c-5c56e5364f13 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.735959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.736571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.736852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.737127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.737751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.739908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.769003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.769686 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.769970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.770242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.770910 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.772827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.854208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1128/2266] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:07.854208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.856776 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.881436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1139/2267] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:07 2022] POST /identity/v3/users => generated 286 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:07.882649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.898746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace88de-b72d-4c16-a269-41b8603a336b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.909294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.911162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.911443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.911721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.912983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.925844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.936915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1140/2268] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:07.937770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:07.942807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25663a4-de7b-4479-8ec9-1b98912ab8cb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.948455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.949350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.949688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.950552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.950667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ebf34-35a5-4aca-b406-73f8ac223abe admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:07.953240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:07.958902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:07.960028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:07.960028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:07.960207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:07.961572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:07.964284 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:07.986258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1141/2269] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:07.987289 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:08.007233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf0599-737f-447a-b301-e6ddb3db28c9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:08.014287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:08.015391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.015391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.015576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.016255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:08.017921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:08.030165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1142/2270] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:08.031098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:08.041255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00c68f-1fe5-47df-b5d6-36d6c09dc181 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:08.047967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:08.049582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:08.049700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:08.050287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.050670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.050955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.051601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:08.053264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:08.060085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1129/2271] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:08.060799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:08.080544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1143/2272] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:08.082510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f2cab-e47d-4514-8930-ec1ba8c63514 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:08.085286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:08.095202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3697daa-e8e4-45b4-918a-4ab114d3a85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:08.096887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3697daa-e8e4-45b4-918a-4ab114d3a85c admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.097525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d819e-d6d7-41db-89d8-b162e188b0a4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:08.098125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3697daa-e8e4-45b4-918a-4ab114d3a85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.099191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3697daa-e8e4-45b4-918a-4ab114d3a85c admin admin] PATH_INFO: `/v3/projects/80f651f1c3994d8eb18a5b5b0c7f4c45/users/5aaa7c209b9c42079c53dbb110b26a81/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.103758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d73ad0-92b9-4e18-a47d-c9d8b4331779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:08.105029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d73ad0-92b9-4e18-a47d-c9d8b4331779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.105029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d73ad0-92b9-4e18-a47d-c9d8b4331779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.105234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d73ad0-92b9-4e18-a47d-c9d8b4331779 admin admin] PATH_INFO: `/v3/projects/d0f6186d2ab14414a1e0a41533a36e29/users/1705522010054ab9895950bd0df202b3/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.123492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3697daa-e8e4-45b4-918a-4ab114d3a85c admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f651f1c3994d8eb18a5b5b0c7f4c45, user_id=5aaa7c209b9c42079c53dbb110b26a81, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:08.126252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d73ad0-92b9-4e18-a47d-c9d8b4331779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f6186d2ab14414a1e0a41533a36e29, user_id=1705522010054ab9895950bd0df202b3, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:08.126554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3697daa-e8e4-45b4-918a-4ab114d3a85c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:08.129832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d73ad0-92b9-4e18-a47d-c9d8b4331779 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:08.267379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1144/2274] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:08 2022] PUT /identity/v3/projects/d0f6186d2ab14414a1e0a41533a36e29/users/1705522010054ab9895950bd0df202b3/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:08.274128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1130/2274] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:08 2022] PUT /identity/v3/projects/80f651f1c3994d8eb18a5b5b0c7f4c45/users/5aaa7c209b9c42079c53dbb110b26a81/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:08.276385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeddb3-362d-43ec-b17a-921c7dd1e6a3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.276772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeddb3-362d-43ec-b17a-921c7dd1e6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.277193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeddb3-362d-43ec-b17a-921c7dd1e6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.278673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.279119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.279546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.317922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-47eeddb3-362d-43ec-b17a-921c7dd1e6a3 None None] MFA Rules not processed for user `5aaa7c209b9c42079c53dbb110b26a81`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:08.319954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] MFA Rules not processed for user `1705522010054ab9895950bd0df202b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:08.320629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-47eeddb3-362d-43ec-b17a-921c7dd1e6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:08.321786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:08.411420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1145/2275] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:08.423217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eeddb3-362d-43ec-b17a-921c7dd1e6a3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:08.457489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1131/2276] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:08.471927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:08.473898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:08.474984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:08.520310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:08.534255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eeddb3-362d-43ec-b17a-921c7dd1e6a3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:08.540438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0a5a9-77de-412c-923d-92056dc03384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:08.541054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a5a9-77de-412c-923d-92056dc03384 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.541380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a5a9-77de-412c-923d-92056dc03384 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.541654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a5a9-77de-412c-923d-92056dc03384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.554335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0a5a9-77de-412c-923d-92056dc03384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:08.555794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0a5a9-77de-412c-923d-92056dc03384 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:08.563064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36053d-9322-4d01-ba10-81c4862c461b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:08.568206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc434-ebc5-4d9c-bc61-8ea74833e81b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:08.569672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc434-ebc5-4d9c-bc61-8ea74833e81b service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.570258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc434-ebc5-4d9c-bc61-8ea74833e81b service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.571040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc434-ebc5-4d9c-bc61-8ea74833e81b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.584418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edc434-ebc5-4d9c-bc61-8ea74833e81b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:08.586093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edc434-ebc5-4d9c-bc61-8ea74833e81b service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:08.597624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1146/2277] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:35:08.599221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1132/2278] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:35:08.821704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664fef-2d4a-463a-a327-fa6ee4c69156 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:08.821983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664fef-2d4a-463a-a327-fa6ee4c69156 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:08.825166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664fef-2d4a-463a-a327-fa6ee4c69156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:08.859711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-48664fef-2d4a-463a-a327-fa6ee4c69156 None None] MFA Rules not processed for user `f67323f672464220a38906660715a9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:08.860694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-48664fef-2d4a-463a-a327-fa6ee4c69156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:08.984810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1147/2279] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:09.273899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:09.388570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:09.572061 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a7c8f-866f-4c78-84a6-22e60b73e881 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:09.572210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a7c8f-866f-4c78-84a6-22e60b73e881 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:09.572541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a7c8f-866f-4c78-84a6-22e60b73e881 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:09.574238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1133/2280] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:35:09.616111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:09.616468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:09.616745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:09.640925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] MFA Rules not processed for user `8547ea522755494ba5177d120561dcff`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:09.642182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:09.744261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1148/2281] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:35:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:09.755499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a7c8f-866f-4c78-84a6-22e60b73e881 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:09.757448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5e1a7c8f-866f-4c78-84a6-22e60b73e881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:09.779140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a7c8f-866f-4c78-84a6-22e60b73e881 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:09.792729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:09.792964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:09.794134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:09.794519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:09.830090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:09.833048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:09.856235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:09.856235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:09.878835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1134/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:09.958305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246b416-7972-4d9a-bd41-3df2b00728fe None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:09.960711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:09.961435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:09.961841 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:09.962173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] PATH_INFO: `/v3/users/d725e4cb658b43438c1f22a80b1dc5ad` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:09.963672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d725e4cb658b43438c1f22a80b1dc5ad)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:09.966444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:10.216964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd725e4cb658b43438c1f22a80b1dc5ad'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:10.332692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd725e4cb658b43438c1f22a80b1dc5ad'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:10.369283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd725e4cb658b43438c1f22a80b1dc5ad'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:10.369916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd725e4cb658b43438c1f22a80b1dc5ad'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:10.370622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6b504b0e-c532-4e78-850b-622500ff04c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd725e4cb658b43438c1f22a80b1dc5ad'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:10.388871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1149/2283] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:09 2022] DELETE /identity/v3/users/d725e4cb658b43438c1f22a80b1dc5ad => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:10.643667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:10.731564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:10.748116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80220415-11e6-419c-ab79-e28fabc60e3b service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:10.851913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:10.874899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:10.878093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:10.878093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:10.878093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] PATH_INFO: `/v3/projects/ce575522168344caacf13cfc7779e25b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:10.878093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce575522168344caacf13cfc7779e25b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:10.879347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:10.999250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce575522168344caacf13cfc7779e25b', 'request_id': 'req-e30488e4-a2a7-4710-a19f-15ad97aa4c52'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:11.006367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce575522168344caacf13cfc7779e25b', 'request_id': 'req-e30488e4-a2a7-4710-a19f-15ad97aa4c52'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:11.088525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce575522168344caacf13cfc7779e25b', 'request_id': 'req-e30488e4-a2a7-4710-a19f-15ad97aa4c52'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:11.101229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] The token cache is being invalidate because project ce575522168344caacf13cfc7779e25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:11.101528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:11.102190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e30488e4-a2a7-4710-a19f-15ad97aa4c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:11.104031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1135/2284] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:10 2022] DELETE /identity/v3/projects/ce575522168344caacf13cfc7779e25b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:11.109712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cf4a-bdc5-4320-b7fa-d18a5c931241 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.109999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cf4a-bdc5-4320-b7fa-d18a5c931241 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.110298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cf4a-bdc5-4320-b7fa-d18a5c931241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.134121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c3b6cf4a-bdc5-4320-b7fa-d18a5c931241 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:11.137647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c3b6cf4a-bdc5-4320-b7fa-d18a5c931241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:11.219153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1150/2285] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:11.227189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c942e2-74ec-40d3-831b-170c1fbf1dce None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.227468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c942e2-74ec-40d3-831b-170c1fbf1dce None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.227740 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c942e2-74ec-40d3-831b-170c1fbf1dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.250029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-50c942e2-74ec-40d3-831b-170c1fbf1dce None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:11.251343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-50c942e2-74ec-40d3-831b-170c1fbf1dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:11.262653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1136/2286] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:11.264388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6cf4a-bdc5-4320-b7fa-d18a5c931241 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:11.274037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6cf4a-bdc5-4320-b7fa-d18a5c931241 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:11.279187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:11.283752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.283752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.283752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.283752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:11.283752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:11.283752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:11.306556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1151/2287] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:11.315523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c942e2-74ec-40d3-831b-170c1fbf1dce None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:11.328371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c942e2-74ec-40d3-831b-170c1fbf1dce None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:11.333279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:11.333873 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.334148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.334448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.335072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:11.336953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:11.449118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1137/2288] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:11 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:11.450465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:11.460835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e91989-2aac-4f70-8b01-5d4188365918 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:11.514617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:11.531061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:11.531897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.532058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.532368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.533667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:11.538959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:11.732870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1152/2289] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:11 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:11.736258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:11.747999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecd1c-bfb9-4847-998b-7d0c5569e3a9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:11.753160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:11.753832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.757083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.757083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.757083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:11.757431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:11.768389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1138/2290] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:11.772451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:11.784119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484c829-966f-4887-986d-dafa483faf82 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:11.789362 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10d458-e521-4e0a-b212-bb2aa2fea69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:11.789975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d458-e521-4e0a-b212-bb2aa2fea69a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.790253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d458-e521-4e0a-b212-bb2aa2fea69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.790693 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d458-e521-4e0a-b212-bb2aa2fea69a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.791374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10d458-e521-4e0a-b212-bb2aa2fea69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:11.792931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10d458-e521-4e0a-b212-bb2aa2fea69a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:11.805272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1153/2291] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:11.805913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:11.809517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:11.820689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91042497-3524-4b66-bd91-8773c2a43e86 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:11.831450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:11.832386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.832989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.833469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] PATH_INFO: `/v3/projects/897030a5075b4ca3ad29d73371306a66/users/55afeedd651248da8ac80c61fb0729cc/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:11.851090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897030a5075b4ca3ad29d73371306a66, user_id=55afeedd651248da8ac80c61fb0729cc, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:11.854472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:11.971631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1139/2292] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:11 2022] PUT /identity/v3/projects/897030a5075b4ca3ad29d73371306a66/users/55afeedd651248da8ac80c61fb0729cc/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:11.977485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:11.978088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:11.979091 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:12.005195 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:12.010260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:12.072799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1154/2293] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:12.083977 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:12.086824 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:12.147690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d686f1f-5a91-4fc5-8b59-ae55adbebf23 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:12.151446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa16c3-7caa-4201-8550-bf9c3999569b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:12.152052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa16c3-7caa-4201-8550-bf9c3999569b service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:12.152348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa16c3-7caa-4201-8550-bf9c3999569b service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:12.153091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa16c3-7caa-4201-8550-bf9c3999569b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:12.164405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa16c3-7caa-4201-8550-bf9c3999569b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:12.166158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa16c3-7caa-4201-8550-bf9c3999569b service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:12.178675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1140/2294] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:12.499432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:12.684289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:12.733041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:14.762805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:14.768881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:14.784156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f517c-6892-42b2-a28c-741b19f7cefd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:14.796978 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:14.797628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:14.797899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:14.798231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] PATH_INFO: `/v3/users/bde4ce181f4942d58e01075e8fb59d60` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:14.803157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde4ce181f4942d58e01075e8fb59d60)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:14.804391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:15.076764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde4ce181f4942d58e01075e8fb59d60'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:15.193065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde4ce181f4942d58e01075e8fb59d60'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:15.214617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde4ce181f4942d58e01075e8fb59d60'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:15.217990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde4ce181f4942d58e01075e8fb59d60'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:15.218658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde4ce181f4942d58e01075e8fb59d60'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:15.244255 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1155/2295] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:14 2022] DELETE /identity/v3/users/bde4ce181f4942d58e01075e8fb59d60 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:15.249939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:15.250547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:15.250547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:15.325513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:15.343086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:15.442392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1141/2296] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:35:15.464460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:15.471017 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:15.528061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:15.589845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b42ed-4317-4fab-9fa4-5e1347b13954 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:15.604687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:15.607262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:15.607612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:15.608222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:15.624623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:15.626737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:15.663392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1156/2297] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:16.678334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:16.687838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:16.730521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394fd66-b9c1-41f1-a8b7-a0215c45395a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:16.753641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:16.754382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:16.754663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:16.754932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] PATH_INFO: `/v3/projects/79b38ce464a043f8a61f9535c7aa18de` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:16.757214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] RBAC: Authorizing `identity:delete_project(project_id=79b38ce464a043f8a61f9535c7aa18de)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:16.765490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:16.918570 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79b38ce464a043f8a61f9535c7aa18de', 'request_id': 'req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:16.920443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:16.928927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79b38ce464a043f8a61f9535c7aa18de', 'request_id': 'req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.029349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79b38ce464a043f8a61f9535c7aa18de', 'request_id': 'req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.046852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] The token cache is being invalidate because project 79b38ce464a043f8a61f9535c7aa18de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:17.047533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.048415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.053430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1142/2298] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:16 2022] DELETE /identity/v3/projects/79b38ce464a043f8a61f9535c7aa18de => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:17.054949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:17.070106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb42210-6d81-4143-b6fc-feab52e275c8 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:17.153014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:17.153650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:17.154044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:17.154423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] PATH_INFO: `/v3/users/1d3fe869d5c74709b01e4d015109a463` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:17.156247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d3fe869d5c74709b01e4d015109a463)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:17.158891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:17.365635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d3fe869d5c74709b01e4d015109a463'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.468110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d3fe869d5c74709b01e4d015109a463'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.483562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d3fe869d5c74709b01e4d015109a463'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.484571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d3fe869d5c74709b01e4d015109a463'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.485391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a9dd9137-79f5-45eb-9b98-b033bccb414c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d3fe869d5c74709b01e4d015109a463'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:17.503228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1157/2299] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:17 2022] DELETE /identity/v3/users/1d3fe869d5c74709b01e4d015109a463 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:18.006991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:18.015673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:18.033561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655c9c8-b75b-4c4d-ada1-67a4a96d074e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:18.183191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:18.185439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:18.185877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:18.186142 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] PATH_INFO: `/v3/projects/cede2dbdc3e14aee9443dddcf4e10fa0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:18.187530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cede2dbdc3e14aee9443dddcf4e10fa0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:18.189919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:18.323277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cede2dbdc3e14aee9443dddcf4e10fa0', 'request_id': 'req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:18.332063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cede2dbdc3e14aee9443dddcf4e10fa0', 'request_id': 'req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:18.416173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cede2dbdc3e14aee9443dddcf4e10fa0', 'request_id': 'req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:18.435676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] The token cache is being invalidate because project cede2dbdc3e14aee9443dddcf4e10fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:18.436491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:18.437414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-9f4fb400-683c-4cc4-b171-ac20ea25dd4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:18.440414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1143/2300] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:18 2022] DELETE /identity/v3/projects/cede2dbdc3e14aee9443dddcf4e10fa0 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:18.445317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb6734-6844-4521-b09b-b52a4fac1c43 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:18.446200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb6734-6844-4521-b09b-b52a4fac1c43 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:18.446681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb6734-6844-4521-b09b-b52a4fac1c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:18.477212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-aaeb6734-6844-4521-b09b-b52a4fac1c43 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:18.478971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aaeb6734-6844-4521-b09b-b52a4fac1c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:18.555374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:18.594621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1158/2301] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:35:18.594823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977e7c-5861-4812-848d-10c12c5993cc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:18.594823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977e7c-5861-4812-848d-10c12c5993cc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:18.594823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977e7c-5861-4812-848d-10c12c5993cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:18.619170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5d977e7c-5861-4812-848d-10c12c5993cc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:18.620821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5d977e7c-5861-4812-848d-10c12c5993cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:18.640238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1144/2302] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:18.644197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb6734-6844-4521-b09b-b52a4fac1c43 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:18.668819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb6734-6844-4521-b09b-b52a4fac1c43 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:18.676583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:18.677640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:18.678076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:18.678744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:18.679606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:18.680206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:18.682113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:18.696864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1159/2303] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:18.701508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977e7c-5861-4812-848d-10c12c5993cc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:18.715603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977e7c-5861-4812-848d-10c12c5993cc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:18.729120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:18.730267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:18.735037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:18.744177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:18.744177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:18.746177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:18.892152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1145/2304] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:18 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:18.896341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:18.916140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a39cd-e266-426a-8cd9-5e2e3714c72d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:19.013375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:19.014608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:19.020735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:19.020735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:19.020735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:19.020735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:19.113113 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1160/2305] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:18 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:19.117306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:19.128456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e2502-025b-48fb-850e-6d56a639166b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:19.133143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:19.133806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:19.134123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:19.138542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:19.138542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:19.138542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:19.149134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1146/2306] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:19.153187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:19.163478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8da6-cead-4aab-8a1f-76e7aa04b345 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:19.169573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7dc4e-6d7b-4343-8806-d49bdc5de493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:19.170220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7dc4e-6d7b-4343-8806-d49bdc5de493 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:19.170548 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7dc4e-6d7b-4343-8806-d49bdc5de493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:19.170822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7dc4e-6d7b-4343-8806-d49bdc5de493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:19.171496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7dc4e-6d7b-4343-8806-d49bdc5de493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:19.173102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7dc4e-6d7b-4343-8806-d49bdc5de493 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:19.188073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1161/2307] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:19.194680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:19.214928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d0f31-f95b-4b3a-a7e4-ebac840d10e1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:19.225720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:19.226447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:19.226735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:19.227013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] PATH_INFO: `/v3/projects/98f247d162394ab2b8a61c43c8182355/users/e94baea8fdb94e9c954780bd59f4018a/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:19.251317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f247d162394ab2b8a61c43c8182355, user_id=e94baea8fdb94e9c954780bd59f4018a, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:19.254074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:19.366055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1147/2308] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:19 2022] PUT /identity/v3/projects/98f247d162394ab2b8a61c43c8182355/users/e94baea8fdb94e9c954780bd59f4018a/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:19.367909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f39c4-0c9a-4304-b2b8-940bc9b782cc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:19.368732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f39c4-0c9a-4304-b2b8-940bc9b782cc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:19.369329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f39c4-0c9a-4304-b2b8-940bc9b782cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:19.449704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-150f39c4-0c9a-4304-b2b8-940bc9b782cc None None] MFA Rules not processed for user `e94baea8fdb94e9c954780bd59f4018a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:19.455365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-150f39c4-0c9a-4304-b2b8-940bc9b782cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:19.632393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1162/2309] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:19.637194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:19.644408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:19.775999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6153-607f-4905-a1d7-f2ff8b33183c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:19.782052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:19.782835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:19.787250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:19.787250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:19.787250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:19.787250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:19.941545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1148/2310] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:19 2022] POST /identity/v3/projects => generated 381 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:19.944565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f39c4-0c9a-4304-b2b8-940bc9b782cc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:19.963198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f39c4-0c9a-4304-b2b8-940bc9b782cc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:20.095523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:20.096142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:20.096420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:20.096694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:20.097338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:20.099312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:20.223488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1163/2311] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:19 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:35:20.233110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:20.249151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a24e07-8a0f-4ce3-a58f-9590fc990858 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:20.285538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:20.285538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:20.285538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:20.285538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:20.286247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:20.286247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:20.306778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:20.308971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1149/2312] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:20.328440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9abd48-d812-4c0a-a5ed-d0741e7f7609 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:20.338116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:20.338844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:20.339124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:20.339398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] PATH_INFO: `/v3/projects/8e823a6534fb479f9b6132cbfc44649b/users/0453c0c60b7f42fcb40f00bf3c5e22a3/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:20.380409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e823a6534fb479f9b6132cbfc44649b, user_id=0453c0c60b7f42fcb40f00bf3c5e22a3, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:20.388393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:20.434173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:20.493323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1164/2313] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:20 2022] PUT /identity/v3/projects/8e823a6534fb479f9b6132cbfc44649b/users/0453c0c60b7f42fcb40f00bf3c5e22a3/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:20.502667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:20.516768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf5dae-81a6-4d17-9ef8-70be9af94c62 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:20.659793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85019a4-792b-455d-92c7-70046c633fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:20.661853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85019a4-792b-455d-92c7-70046c633fce admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:20.663338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85019a4-792b-455d-92c7-70046c633fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:20.663854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85019a4-792b-455d-92c7-70046c633fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:20.664714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85019a4-792b-455d-92c7-70046c633fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:20.671527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85019a4-792b-455d-92c7-70046c633fce admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:20.727408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1150/2314] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:20.729469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:20.756267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e72010-327c-4885-9772-2dac267850e8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:20.773288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8dae-d185-4c25-b93e-b36117d054b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:20.774609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8dae-d185-4c25-b93e-b36117d054b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:20.775050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8dae-d185-4c25-b93e-b36117d054b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:20.790951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8dae-d185-4c25-b93e-b36117d054b1 admin admin] PATH_INFO: `/v3/projects/8e823a6534fb479f9b6132cbfc44649b/users/0453c0c60b7f42fcb40f00bf3c5e22a3/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:20.796337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf8dae-d185-4c25-b93e-b36117d054b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e823a6534fb479f9b6132cbfc44649b, user_id=0453c0c60b7f42fcb40f00bf3c5e22a3, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:20.804750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf8dae-d185-4c25-b93e-b36117d054b1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:20.850078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1165/2315] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:20 2022] PUT /identity/v3/projects/8e823a6534fb479f9b6132cbfc44649b/users/0453c0c60b7f42fcb40f00bf3c5e22a3/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:20.863186 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:20.863898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:20.864119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:20.911630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] MFA Rules not processed for user `0453c0c60b7f42fcb40f00bf3c5e22a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:20.916164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:20.964458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:20.964779 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:20.965083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:21.051319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] MFA Rules not processed for user `55afeedd651248da8ac80c61fb0729cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:21.056915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:21.125629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1151/2316] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:21.139879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:21.147848 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:21.202299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1166/2317] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:21.213037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:21.216250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:21.237174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea2ded-95bb-4648-97aa-b513028cbb4f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:21.242162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:21.243000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:21.243482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:21.244160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:21.267659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:21.269128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:21.286550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1152/2318] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:21.318829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04443a73-4b3b-455e-9786-82ab1bd24149 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:21.324835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f09d2-facf-49c5-822d-0c91330cc106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:21.325399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f09d2-facf-49c5-822d-0c91330cc106 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:21.325672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f09d2-facf-49c5-822d-0c91330cc106 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:21.325943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f09d2-facf-49c5-822d-0c91330cc106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:21.335212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f09d2-facf-49c5-822d-0c91330cc106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:21.336843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f09d2-facf-49c5-822d-0c91330cc106 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:21.347354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1167/2319] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:21.526191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:22.083903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:22.161162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf83c-8af6-4baf-890e-2b7fcf64555b service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:22.171745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:22.172488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:22.172651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:22.172926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] PATH_INFO: `/v3/users/e94baea8fdb94e9c954780bd59f4018a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:22.175040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e94baea8fdb94e9c954780bd59f4018a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:22.176981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:22.409077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e94baea8fdb94e9c954780bd59f4018a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:22.477436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e94baea8fdb94e9c954780bd59f4018a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:22.490637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e94baea8fdb94e9c954780bd59f4018a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:22.497834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e94baea8fdb94e9c954780bd59f4018a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:22.498588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e94baea8fdb94e9c954780bd59f4018a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:22.504744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1153/2320] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:22 2022] DELETE /identity/v3/users/e94baea8fdb94e9c954780bd59f4018a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:22.509424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:22.509906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:22.510155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:22.530821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:22.532116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:22.537655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:22.594835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1168/2321] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:22.604364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:22.606050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:22.669290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6692d7-ed5e-4bd9-a374-2bbbdf2fad4d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:22.673016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:22.673617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:22.673922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:22.674194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:22.683719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:22.684951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:22.694963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1154/2322] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:23.190042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:23.192159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:23.203180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:23.218243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c82ca-c720-4e02-abf9-be7b3979f6f2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:23.224528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:23.225488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:23.225776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:23.226048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] PATH_INFO: `/v3/projects/98f247d162394ab2b8a61c43c8182355` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:23.227272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f247d162394ab2b8a61c43c8182355)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:23.229253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:23.364926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f247d162394ab2b8a61c43c8182355', 'request_id': 'req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:23.440260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f247d162394ab2b8a61c43c8182355', 'request_id': 'req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:23.454153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f247d162394ab2b8a61c43c8182355', 'request_id': 'req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:23.486456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] The token cache is being invalidate because project 98f247d162394ab2b8a61c43c8182355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:23.487002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:23.489034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09bfb4cc-ebd7-49b4-b289-9e8ba62f80a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:23.491176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1169/2323] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:23 2022] DELETE /identity/v3/projects/98f247d162394ab2b8a61c43c8182355 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:23.495314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:23.510101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c43c71-7db7-46ff-8e86-6e2797d54c9b service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:23.588699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:23.590412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:23.591081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:23.591503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] PATH_INFO: `/v3/users/0453c0c60b7f42fcb40f00bf3c5e22a3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:23.593120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0453c0c60b7f42fcb40f00bf3c5e22a3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:23.595556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:23.846673 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0453c0c60b7f42fcb40f00bf3c5e22a3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:24.021976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0453c0c60b7f42fcb40f00bf3c5e22a3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:24.036643 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0453c0c60b7f42fcb40f00bf3c5e22a3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:24.049146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0453c0c60b7f42fcb40f00bf3c5e22a3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:24.049999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0453c0c60b7f42fcb40f00bf3c5e22a3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:24.056432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1155/2324] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:23 2022] DELETE /identity/v3/users/0453c0c60b7f42fcb40f00bf3c5e22a3 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:24.533052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866198e4-568b-412d-8063-31552ed6a668 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:24.533538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866198e4-568b-412d-8063-31552ed6a668 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:24.533630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866198e4-568b-412d-8063-31552ed6a668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:24.574573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-866198e4-568b-412d-8063-31552ed6a668 None None] MFA Rules not processed for user `55afeedd651248da8ac80c61fb0729cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:24.575882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-866198e4-568b-412d-8063-31552ed6a668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:24.620663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:24.623341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:24.687665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1170/2325] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:24.784726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90c8d1-7b57-4e33-aa5b-8407f9363328 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:24.793726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:24.793726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:24.793726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:24.793726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] PATH_INFO: `/v3/projects/8e823a6534fb479f9b6132cbfc44649b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:24.795176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e823a6534fb479f9b6132cbfc44649b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:24.797735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:24.990548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e823a6534fb479f9b6132cbfc44649b', 'request_id': 'req-6c60e79f-2803-4e3c-ae92-7759453e1f1d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:24.998648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e823a6534fb479f9b6132cbfc44649b', 'request_id': 'req-6c60e79f-2803-4e3c-ae92-7759453e1f1d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:25.081889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e823a6534fb479f9b6132cbfc44649b', 'request_id': 'req-6c60e79f-2803-4e3c-ae92-7759453e1f1d'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:25.105836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] The token cache is being invalidate because project 8e823a6534fb479f9b6132cbfc44649b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:25.106265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:25.108277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-6c60e79f-2803-4e3c-ae92-7759453e1f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:25.110373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1156/2326] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:24 2022] DELETE /identity/v3/projects/8e823a6534fb479f9b6132cbfc44649b => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:25.115333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0c0c-08ac-4313-bf30-3f2ddd05a5b5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.115613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0c0c-08ac-4313-bf30-3f2ddd05a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.115881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0c0c-08ac-4313-bf30-3f2ddd05a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.143687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8f8a0c0c-08ac-4313-bf30-3f2ddd05a5b5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:25.144823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8f8a0c0c-08ac-4313-bf30-3f2ddd05a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:25.197166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:25.235701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1171/2327] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:25.243437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28f9f4-5b50-4968-8fef-78dfab0811be None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.246190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28f9f4-5b50-4968-8fef-78dfab0811be None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.247049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28f9f4-5b50-4968-8fef-78dfab0811be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.299999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aa28f9f4-5b50-4968-8fef-78dfab0811be None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:25.302918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aa28f9f4-5b50-4968-8fef-78dfab0811be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:25.334346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1157/2328] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:35:25.340604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a0c0c-08ac-4313-bf30-3f2ddd05a5b5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.351722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a0c0c-08ac-4313-bf30-3f2ddd05a5b5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.357004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.357923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.358200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.358508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.359160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:25.359473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.361234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.380967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1172/2329] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:25.385171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28f9f4-5b50-4968-8fef-78dfab0811be None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.395688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28f9f4-5b50-4968-8fef-78dfab0811be None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.400701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:25.403842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.404412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.404756 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.405045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.405686 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.408068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.503121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1158/2330] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:25 2022] POST /identity/v3/projects => generated 359 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:25.507006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.519823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa0ac-092c-4a2c-87b9-0d8842ee23b0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.588482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.590377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.591041 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.591518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.592163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.594142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.656143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.657900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:25.682949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1173/2331] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:25 2022] POST /identity/v3/users => generated 280 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:25.689070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.699937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb74f8-aaff-4f8b-b3df-23b75acb28f1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.705106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.705701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.706148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.706624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.707292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.709198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.719035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6d627-95b2-48d3-aeec-ee62bd325198 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.721591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1174/2332] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:25.724823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.724823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.724823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.731208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.731348 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.731488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:25.736478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ecd1-fcb3-474d-825a-085ede093eac admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.741226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.741876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.742221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.742583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.743285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.745137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.756536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1175/2333] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:25.760586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.770795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c04494-e8f0-4a3d-ad83-e555bb1bcdb2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.776324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a39a4-dd13-4bbc-b85f-db4cfc5a25ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.776968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a39a4-dd13-4bbc-b85f-db4cfc5a25ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.777767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a39a4-dd13-4bbc-b85f-db4cfc5a25ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.778935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a39a4-dd13-4bbc-b85f-db4cfc5a25ac admin admin] PATH_INFO: `/v3/projects/4a56278d98e24835bcdbe5be29befdcf/users/fc3a99d2f26d46a590a945503b178b9d/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.799853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a39a4-dd13-4bbc-b85f-db4cfc5a25ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a56278d98e24835bcdbe5be29befdcf, user_id=fc3a99d2f26d46a590a945503b178b9d, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.802698 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a39a4-dd13-4bbc-b85f-db4cfc5a25ac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.811341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.813098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.826587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1159/2334] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:35:25.947521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.950172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:25.954463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1176/2335] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:25 2022] PUT /identity/v3/projects/4a56278d98e24835bcdbe5be29befdcf/users/fc3a99d2f26d46a590a945503b178b9d/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:25.965001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.966299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.966299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.981155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6212c-38b3-43f4-b07f-e0344a24d29f service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:26.016675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] MFA Rules not processed for user `fc3a99d2f26d46a590a945503b178b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:26.017906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:26.108332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:26.109053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:26.109473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:26.109958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] PATH_INFO: `/v3/users/5aaa7c209b9c42079c53dbb110b26a81` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:26.117909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aaa7c209b9c42079c53dbb110b26a81)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:26.118746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:26.177815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1177/2336] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:35:26.220603 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:26.223480 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:26.290183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c78d1-af69-4c0f-b3c1-c8224458574b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:26.294691 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:26.295297 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:26.295576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:26.295849 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:26.304517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:26.306014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:26.316572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1178/2337] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:26.333887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:26.358593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aaa7c209b9c42079c53dbb110b26a81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.374530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3562715-e5e5-428e-a9d9-ab6ad9e261e4 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:26.381244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:26.381882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:26.382164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:26.382512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] PATH_INFO: `/v3/users/fc3a99d2f26d46a590a945503b178b9d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:26.383994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3a99d2f26d46a590a945503b178b9d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:26.385563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:26.478876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aaa7c209b9c42079c53dbb110b26a81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.492767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aaa7c209b9c42079c53dbb110b26a81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.499956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aaa7c209b9c42079c53dbb110b26a81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.500806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b0b1aca2-b959-46c3-9b77-0534d655951d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aaa7c209b9c42079c53dbb110b26a81'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.508434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1160/2338] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:25 2022] DELETE /identity/v3/users/5aaa7c209b9c42079c53dbb110b26a81 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:26.513602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:26.513880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:26.514163 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:26.535799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:26.540530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:26.605436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1161/2339] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:26.616847 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:26.619443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:26.649792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3a99d2f26d46a590a945503b178b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.689969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa45b3c-1edf-4fe6-9c87-a7695d9e98cb None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:26.694989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b204bee-4805-4e35-bd04-85fdcb397a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:26.694989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b204bee-4805-4e35-bd04-85fdcb397a2c service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:26.694989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b204bee-4805-4e35-bd04-85fdcb397a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:26.695575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b204bee-4805-4e35-bd04-85fdcb397a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:26.706225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b204bee-4805-4e35-bd04-85fdcb397a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:26.708583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b204bee-4805-4e35-bd04-85fdcb397a2c service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:26.719515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1162/2340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:35:26.744734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3a99d2f26d46a590a945503b178b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.761088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3a99d2f26d46a590a945503b178b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.762345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3a99d2f26d46a590a945503b178b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.763610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3a99d2f26d46a590a945503b178b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:26.781969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1179/2341] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:26 2022] DELETE /identity/v3/users/fc3a99d2f26d46a590a945503b178b9d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:26.788245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:26.788547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:26.788960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:26.814460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:26.815818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:26.909172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1163/2342] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:26.929192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:26.937336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:27.022514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000ca0-d57b-4210-97bc-ad3ebf2e0021 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:27.032859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:27.033492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:27.034772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:27.035286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:27.046476 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:27.048182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:27.059212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1180/2343] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:27.290954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:27.368976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:27.564680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:27.566669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:27.588648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c1820-2278-4699-a7c5-f0d6ee8a6101 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:27.600083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:27.601098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:27.601219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:27.602133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] PATH_INFO: `/v3/projects/80f651f1c3994d8eb18a5b5b0c7f4c45` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:27.603050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f651f1c3994d8eb18a5b5b0c7f4c45)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:27.604976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:27.671295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:27.675266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:27.695610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245fe9f-998e-4448-8e6e-3b8c81123da9 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:27.702976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:27.704571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:27.705577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:27.706988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] PATH_INFO: `/v3/projects/4a56278d98e24835bcdbe5be29befdcf` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:27.708442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a56278d98e24835bcdbe5be29befdcf)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:27.711080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:27.742187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f651f1c3994d8eb18a5b5b0c7f4c45', 'request_id': 'req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.749329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f651f1c3994d8eb18a5b5b0c7f4c45', 'request_id': 'req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.815211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f651f1c3994d8eb18a5b5b0c7f4c45', 'request_id': 'req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.832214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] The token cache is being invalidate because project 80f651f1c3994d8eb18a5b5b0c7f4c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:27.832513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.835681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.840761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1164/2344] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:27 2022] DELETE /identity/v3/projects/80f651f1c3994d8eb18a5b5b0c7f4c45 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:27.840761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:27.842164 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:27.842764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a56278d98e24835bcdbe5be29befdcf', 'request_id': 'req-0eadc459-357a-45ad-bb15-936c5f51000d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.918770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a56278d98e24835bcdbe5be29befdcf', 'request_id': 'req-0eadc459-357a-45ad-bb15-936c5f51000d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.930831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a56278d98e24835bcdbe5be29befdcf', 'request_id': 'req-0eadc459-357a-45ad-bb15-936c5f51000d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.943316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7183-771f-4719-8e4e-beb2d67b9d3b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:27.947174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] The token cache is being invalidate because project 4a56278d98e24835bcdbe5be29befdcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:27.947471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.948239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0eadc459-357a-45ad-bb15-936c5f51000d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:27.950064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1181/2345] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:27 2022] DELETE /identity/v3/projects/4a56278d98e24835bcdbe5be29befdcf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:27.951344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6efd-3f3a-4423-a06d-cea34ec5c1c3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:27.951747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6efd-3f3a-4423-a06d-cea34ec5c1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:27.952021 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6efd-3f3a-4423-a06d-cea34ec5c1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:27.974432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3a3d6efd-3f3a-4423-a06d-cea34ec5c1c3 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:27.975702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3a3d6efd-3f3a-4423-a06d-cea34ec5c1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:27.988667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:28.025684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3e20d-70e7-403f-82f3-37aa6883e60f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.026677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3e20d-70e7-403f-82f3-37aa6883e60f service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.027065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3e20d-70e7-403f-82f3-37aa6883e60f service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.028093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3e20d-70e7-403f-82f3-37aa6883e60f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.030250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-58f3e20d-70e7-403f-82f3-37aa6883e60f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:28.045232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3e20d-70e7-403f-82f3-37aa6883e60f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.046880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3e20d-70e7-403f-82f3-37aa6883e60f service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.066016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1165/2346] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:28.068165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821de2c-fad2-41b2-a43c-f73d8f971021 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.068457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821de2c-fad2-41b2-a43c-f73d8f971021 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.068732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821de2c-fad2-41b2-a43c-f73d8f971021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.068912 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1182/2347] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:28.085622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016455-58c5-4052-bf63-b2c320b6700c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.087062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016455-58c5-4052-bf63-b2c320b6700c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.087361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016455-58c5-4052-bf63-b2c320b6700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.098107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5821de2c-fad2-41b2-a43c-f73d8f971021 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:28.099390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5821de2c-fad2-41b2-a43c-f73d8f971021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:28.112747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1166/2348] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:28.125454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63526f9-c4d1-449a-a5ae-3f504c08ce34 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.126796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63526f9-c4d1-449a-a5ae-3f504c08ce34 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.127129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63526f9-c4d1-449a-a5ae-3f504c08ce34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.134699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b8016455-58c5-4052-bf63-b2c320b6700c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:28.138309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b8016455-58c5-4052-bf63-b2c320b6700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:28.152021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d63526f9-c4d1-449a-a5ae-3f504c08ce34 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:28.153839 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d63526f9-c4d1-449a-a5ae-3f504c08ce34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:28.165114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1167/2349] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:28.167105 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1183/2350] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:28.172763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63526f9-c4d1-449a-a5ae-3f504c08ce34 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.174977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8016455-58c5-4052-bf63-b2c320b6700c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.191734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63526f9-c4d1-449a-a5ae-3f504c08ce34 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.195714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8016455-58c5-4052-bf63-b2c320b6700c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.196585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.197241 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.197522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.197789 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.198729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:28.199045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.201849 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.211892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.212778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.213276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.213719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.214521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:35:28.215484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.217904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1168/2351] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:35:28.222733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.226420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.235765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee9980-b55a-4b89-af74-d34aedeb2008 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.243640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.244714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.245067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.245437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.246087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.248942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.261374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1184/2352] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:35:28.267034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.279055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdb1eb-6016-478e-896e-75a363e5ffba admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.284064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.284694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.284982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.285251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.285870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.287814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.341915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1169/2353] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:35:28.345659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.351456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1185/2354] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:35:28.355524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.396805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:28.427587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c161-b956-455e-871c-05bbda7cb6fe admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.434303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8b566-e298-4823-a254-6eb0294180d6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.437174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.437972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.438453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.438983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.439790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.441975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.443567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.444187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.444511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.445504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.446155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.448256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.530833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1170/2355] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:35:28.534705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.540705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1186/2356] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:28.545291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.548602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8281e16-c19b-4970-a76a-5534cf50e40b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.556679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828253ac-edde-4554-97b7-ec2915db9549 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.557563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.558330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.558798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.559080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.559862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.561640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.562095 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.562274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.562608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.562886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.563758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.565733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.573270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1171/2357] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:28.577607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.585893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1187/2358] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:28.588042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8182bd-3b1e-4c06-b192-c3c450237fa0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.589071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.593667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.594280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.594593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.594872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.595515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.597172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.601171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551476f-15cf-4f85-8705-66f6f60d39ba admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.606432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.608174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.608764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.608902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1172/2359] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:28.610621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.611330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.612935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.614988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.626689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e2a7a-99e5-4a7b-b188-f9b2f676dc25 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.634248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1188/2360] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:28.638027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3aab31-ec1e-4040-adff-b91b9ef394e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.638187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.638287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3aab31-ec1e-4040-adff-b91b9ef394e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.638287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3aab31-ec1e-4040-adff-b91b9ef394e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.638287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3aab31-ec1e-4040-adff-b91b9ef394e1 admin admin] PATH_INFO: `/v3/projects/e069855c07d94296a331d49a7a2a7ce4/users/0fe07f55f4d54059a193d4611e0a4f92/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.648743 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f19d2e-665e-48d7-bbe2-ed141708483c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:28.655343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dc999-a713-4d0f-af80-83d7c72a73d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:28.656252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc999-a713-4d0f-af80-83d7c72a73d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.656825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc999-a713-4d0f-af80-83d7c72a73d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.657285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc999-a713-4d0f-af80-83d7c72a73d5 admin admin] PATH_INFO: `/v3/projects/e7cfd5293fa74a9094df6214d8f077d6/users/89aed92f4ca6453ca113e52442c8c4e2/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.665617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3aab31-ec1e-4040-adff-b91b9ef394e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e069855c07d94296a331d49a7a2a7ce4, user_id=0fe07f55f4d54059a193d4611e0a4f92, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.668965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3aab31-ec1e-4040-adff-b91b9ef394e1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.676997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dc999-a713-4d0f-af80-83d7c72a73d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7cfd5293fa74a9094df6214d8f077d6, user_id=89aed92f4ca6453ca113e52442c8c4e2, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:28.680503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dc999-a713-4d0f-af80-83d7c72a73d5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:28.753063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1173/2361] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:28 2022] PUT /identity/v3/projects/e069855c07d94296a331d49a7a2a7ce4/users/0fe07f55f4d54059a193d4611e0a4f92/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:28.757229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.757578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.757939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.759193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1189/2362] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:28 2022] PUT /identity/v3/projects/e7cfd5293fa74a9094df6214d8f077d6/users/89aed92f4ca6453ca113e52442c8c4e2/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:28.799370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:28.800717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:28.849280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d88ae-c032-4d0d-bc6a-1db7fddc5dd2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:28.849618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d88ae-c032-4d0d-bc6a-1db7fddc5dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:28.849821 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d88ae-c032-4d0d-bc6a-1db7fddc5dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:28.884521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-718d88ae-c032-4d0d-bc6a-1db7fddc5dd2 None None] MFA Rules not processed for user `89aed92f4ca6453ca113e52442c8c4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:28.887935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-718d88ae-c032-4d0d-bc6a-1db7fddc5dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:28.949845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1174/2363] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:28.957677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:28.959289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:28.986031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:28.996890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1190/2364] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:35:29.001338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d88ae-c032-4d0d-bc6a-1db7fddc5dd2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:29.021639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d88ae-c032-4d0d-bc6a-1db7fddc5dd2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:29.028962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:29.029592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:29.029874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:29.030188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:29.030989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:29.033545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:29.040474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbd9a9-a0be-4789-9cca-1254a22920a8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:29.045268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:29.045798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:29.046159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:29.046577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:29.056736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:29.058258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:29.070131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1175/2365] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:35:29.123064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1191/2366] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:29 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:29.126894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:29.142310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eb4b0-df87-4a14-bd8d-8d3001a663de service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:29.218426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:29.218946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:29.219233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:29.219500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:29.220217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:29.222983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:29.230146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:29.407832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1176/2367] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:29 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:29.411907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:29.423356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c3cd1-c6ec-46c1-9d0e-e4733d85fa53 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:29.428412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:29.429025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:29.429307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:29.429741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:29.430422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:29.432055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:29.444271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1192/2368] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:29.448160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:29.460775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10145b-a1be-463c-bc4e-d0ba67817fa5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:29.477761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d1196-dd01-4489-8988-a6f21350474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:29.483310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d1196-dd01-4489-8988-a6f21350474d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:29.483310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d1196-dd01-4489-8988-a6f21350474d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:29.483310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d1196-dd01-4489-8988-a6f21350474d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:29.483310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d1196-dd01-4489-8988-a6f21350474d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:29.483710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d1196-dd01-4489-8988-a6f21350474d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:29.506214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1177/2369] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:35:29.507968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:29.523652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce52d7-86cb-4736-97ed-d26462b1a345 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:29.547829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:29.548513 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:29.548795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:29.549070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] PATH_INFO: `/v3/projects/df958a944ae44a89a751341b9ac8cba6/users/47f1a4849f29495188322fac88e1c618/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:29.575465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df958a944ae44a89a751341b9ac8cba6, user_id=47f1a4849f29495188322fac88e1c618, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:29.579029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:29.725019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1193/2370] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:29 2022] PUT /identity/v3/projects/df958a944ae44a89a751341b9ac8cba6/users/47f1a4849f29495188322fac88e1c618/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:29.737414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7227-a771-4c51-a2d0-75dc44a7dfb8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:29.737744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7227-a771-4c51-a2d0-75dc44a7dfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:29.738029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7227-a771-4c51-a2d0-75dc44a7dfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:29.771933 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-153a7227-a771-4c51-a2d0-75dc44a7dfb8 None None] MFA Rules not processed for user `47f1a4849f29495188322fac88e1c618`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:29.773260 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-153a7227-a771-4c51-a2d0-75dc44a7dfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:29.928267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1178/2371] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:29.933324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:30.014003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:30.096013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aca0fd-c233-45ef-b27f-b3a6306a4b96 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:30.102983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:30.103656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:30.103936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:30.108261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:30.108261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:30.108261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:30.218521 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1194/2372] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:29 2022] POST /identity/v3/projects => generated 377 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:30.223782 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7227-a771-4c51-a2d0-75dc44a7dfb8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:30.238434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7227-a771-4c51-a2d0-75dc44a7dfb8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:30.325406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:30.326569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:30.327229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:30.327616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:30.328387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:30.343755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:30.469608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1179/2373] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:30 2022] POST /identity/v3/users => generated 295 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:30.472342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:30.493240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7bbae-e173-4ddc-bfed-4a6865c0bb22 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:30.501818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:30.502533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:30.502815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:30.503086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:30.503758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:30.505376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:30.528995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1195/2374] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:30.533138 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:30.548855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2cc5b-3a34-41f5-822c-6272a56675ee admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:30.564434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:30.565370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:30.565370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:30.565715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] PATH_INFO: `/v3/projects/c452ddf22149433ba6d90bad3e05678f/users/bd89a83221e6405591c917f96aeffd0b/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:30.597925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c452ddf22149433ba6d90bad3e05678f, user_id=bd89a83221e6405591c917f96aeffd0b, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:30.600844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:30.738043 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1180/2375] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:30 2022] PUT /identity/v3/projects/c452ddf22149433ba6d90bad3e05678f/users/bd89a83221e6405591c917f96aeffd0b/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:30.739832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:30.756635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70253a16-248c-4b72-8d64-ea6261a6e15d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:30.856326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd6fc-6414-4c08-b56f-cd39582bcaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:30.857216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd6fc-6414-4c08-b56f-cd39582bcaf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:30.857700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd6fc-6414-4c08-b56f-cd39582bcaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:30.858127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd6fc-6414-4c08-b56f-cd39582bcaf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:30.859074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd6fc-6414-4c08-b56f-cd39582bcaf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:30.861749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd6fc-6414-4c08-b56f-cd39582bcaf8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:30.886443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1196/2376] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:30.888199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:30.901866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb94307-7fd7-4770-aec3-853456baee6a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:30.908461 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:30.909757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:30.910321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:30.910835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] PATH_INFO: `/v3/projects/c452ddf22149433ba6d90bad3e05678f/users/bd89a83221e6405591c917f96aeffd0b/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:30.919623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c452ddf22149433ba6d90bad3e05678f, user_id=bd89a83221e6405591c917f96aeffd0b, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:30.935658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:30.939860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:30.951407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1181/2377] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:30 2022] PUT /identity/v3/projects/c452ddf22149433ba6d90bad3e05678f/users/bd89a83221e6405591c917f96aeffd0b/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:30.964266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c595-cc6a-4c3a-a86b-276387f84188 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:30.964690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c595-cc6a-4c3a-a86b-276387f84188 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:30.965444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c595-cc6a-4c3a-a86b-276387f84188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:31.006573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9176c595-cc6a-4c3a-a86b-276387f84188 None None] MFA Rules not processed for user `bd89a83221e6405591c917f96aeffd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:31.009247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9176c595-cc6a-4c3a-a86b-276387f84188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:31.146084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1197/2378] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:31.159812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:31.161871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:31.270102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef22877-0646-41f0-8390-cc47af1ceb7a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:31.283595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:31.284687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:31.285248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:31.285750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:31.326569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:31.391719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:31.393542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:31.412385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1182/2379] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:32.140901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:34.876092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176c595-cc6a-4c3a-a86b-276387f84188 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:34.893546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176c595-cc6a-4c3a-a86b-276387f84188 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:34.900055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5d827-34a9-4db1-bb6d-979161bfb2e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:34.900870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5d827-34a9-4db1-bb6d-979161bfb2e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:34.903057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5d827-34a9-4db1-bb6d-979161bfb2e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:34.903624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5d827-34a9-4db1-bb6d-979161bfb2e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:34.970020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:35.015106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5d827-34a9-4db1-bb6d-979161bfb2e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:35.016825 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5d827-34a9-4db1-bb6d-979161bfb2e7 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:35.032302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1198/2380] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:36.935494 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:36.948399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec634-6b09-4141-855c-f4bc447a07c0 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:36.955118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:36.955971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:36.956415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:36.957338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:36.976936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:36.981402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:37.003079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1183/2381] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:38.616592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe808c4-1c92-46ca-a85e-daadcb2b76a5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:38.617043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe808c4-1c92-46ca-a85e-daadcb2b76a5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:38.617122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe808c4-1c92-46ca-a85e-daadcb2b76a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:38.654474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bfe808c4-1c92-46ca-a85e-daadcb2b76a5 None None] MFA Rules not processed for user `0fe07f55f4d54059a193d4611e0a4f92`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:38.654474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bfe808c4-1c92-46ca-a85e-daadcb2b76a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:38.763486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1199/2382] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:38.767686 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:38.893846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef9fcb-bcca-4982-a3d0-18c4cf66c9b9 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:38.899625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:38.900245 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:38.900591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:38.900867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:38.901732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:38.904805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:39.034259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1184/2383] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:35:38 2022] POST /identity/v3/projects => generated 371 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:35:39.035410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe808c4-1c92-46ca-a85e-daadcb2b76a5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:39.052192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe808c4-1c92-46ca-a85e-daadcb2b76a5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:39.135987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:39.137364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:39.137749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:39.138423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:39.139277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:39.141671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:39.284592 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1200/2384] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:35:39 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:35:39.288936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:39.302836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167837a-30fb-4b29-ad48-fd4fc49ec3a6 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:39.308429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:39.309144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:39.309491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:39.309830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:39.310642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:39.312386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:39.325827 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1185/2385] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:39.330735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:39.342580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bda7d2-5e58-4afe-a162-44067f3ff43e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:39.353919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:39.353919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:39.353919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:39.353919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] PATH_INFO: `/v3/projects/cfe4d8665af44a05b457b11d810a13b4/users/4f791616148c4fa08c83c699b0a21077/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:39.374644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe4d8665af44a05b457b11d810a13b4, user_id=4f791616148c4fa08c83c699b0a21077, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:39.378158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:39.473181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1201/2386] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:39 2022] PUT /identity/v3/projects/cfe4d8665af44a05b457b11d810a13b4/users/4f791616148c4fa08c83c699b0a21077/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:35:39.477105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:39.488617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709c2f-7947-4111-8e4f-841a76e2b34f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:39.535505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:39.551490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3e820-5f98-4569-afbd-db410ed22796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:39.552433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3e820-5f98-4569-afbd-db410ed22796 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:39.552879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3e820-5f98-4569-afbd-db410ed22796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:39.559054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3e820-5f98-4569-afbd-db410ed22796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:39.559054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3e820-5f98-4569-afbd-db410ed22796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:39.559054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3e820-5f98-4569-afbd-db410ed22796 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:39.575612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1186/2387] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:35:39.580736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:39.605455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01267fb3-f26b-4b1e-b922-d5fdf46c1560 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:39.614780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:39.616488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:39.616488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:39.619232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] PATH_INFO: `/v3/projects/cfe4d8665af44a05b457b11d810a13b4/users/4f791616148c4fa08c83c699b0a21077/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:39.620188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe4d8665af44a05b457b11d810a13b4, user_id=4f791616148c4fa08c83c699b0a21077, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:39.627704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:39.643063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1202/2388] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:35:39 2022] PUT /identity/v3/projects/cfe4d8665af44a05b457b11d810a13b4/users/4f791616148c4fa08c83c699b0a21077/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:41.599718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844c0a-ad03-4838-b821-3a67fa0c799f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:41.600382 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844c0a-ad03-4838-b821-3a67fa0c799f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:41.600805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844c0a-ad03-4838-b821-3a67fa0c799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:41.628286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-25844c0a-ad03-4838-b821-3a67fa0c799f None None] MFA Rules not processed for user `f6fb96d9442147e0a2201fe890d6f955`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:41.629743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-25844c0a-ad03-4838-b821-3a67fa0c799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:41.728099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1187/2389] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:43.254276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:43.279526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:43.347032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea20cb-9edd-4924-b6e2-eea1cfb7bd29 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:43.352659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:43.353884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:43.354315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:43.354786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] PATH_INFO: `/v3/users/89aed92f4ca6453ca113e52442c8c4e2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:43.358492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] RBAC: Authorizing `identity:delete_user(user_id=89aed92f4ca6453ca113e52442c8c4e2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:43.361165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:43.632951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89aed92f4ca6453ca113e52442c8c4e2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:43.723983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89aed92f4ca6453ca113e52442c8c4e2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:43.748882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89aed92f4ca6453ca113e52442c8c4e2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:43.749970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89aed92f4ca6453ca113e52442c8c4e2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:43.750833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89aed92f4ca6453ca113e52442c8c4e2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:43.791510 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1203/2390] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:43 2022] DELETE /identity/v3/users/89aed92f4ca6453ca113e52442c8c4e2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:43.796825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:43.797321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:43.797444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:43.819191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:43.820858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:43.925270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1188/2391] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:43.932448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:43.934321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:44.016187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39036aef-3399-48c1-8eb8-f4a19e5c772c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:44.020469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:44.021589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:44.021589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:44.021812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:44.034164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:44.035616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:44.087031 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1204/2392] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:44.814124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:44.816634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:44.835932 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9b5d9-876c-4aa6-9542-446c8bcc2fe9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:44.843762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:44.844667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:44.845372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:44.845807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] PATH_INFO: `/v3/projects/e7cfd5293fa74a9094df6214d8f077d6` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:44.847178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7cfd5293fa74a9094df6214d8f077d6)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:44.870035 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:45.068480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7cfd5293fa74a9094df6214d8f077d6', 'request_id': 'req-006910c2-533c-401d-a09a-e3f4c0bb0de2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.083115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7cfd5293fa74a9094df6214d8f077d6', 'request_id': 'req-006910c2-533c-401d-a09a-e3f4c0bb0de2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.154753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7cfd5293fa74a9094df6214d8f077d6', 'request_id': 'req-006910c2-533c-401d-a09a-e3f4c0bb0de2'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.171928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] The token cache is being invalidate because project e7cfd5293fa74a9094df6214d8f077d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:45.172292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.173381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.175188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1189/2393] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:44 2022] DELETE /identity/v3/projects/e7cfd5293fa74a9094df6214d8f077d6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:45.179325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:45.191889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36886af4-6232-466f-b475-31c2732f8b9d service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:45.235485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:45.267124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:45.285813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:45.285813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:45.285813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] PATH_INFO: `/v3/users/47f1a4849f29495188322fac88e1c618` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:45.285813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f1a4849f29495188322fac88e1c618)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:45.285813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:45.492774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f1a4849f29495188322fac88e1c618'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.563202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f1a4849f29495188322fac88e1c618'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.578220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f1a4849f29495188322fac88e1c618'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.579058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f1a4849f29495188322fac88e1c618'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.579996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f1a4849f29495188322fac88e1c618'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:45.658422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1205/2394] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:45 2022] DELETE /identity/v3/users/47f1a4849f29495188322fac88e1c618 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:46.197531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:46.200505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:46.240911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006910c2-533c-401d-a09a-e3f4c0bb0de2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:46.265309 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:46.343133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:46.343790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:46.344074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:46.344378 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] PATH_INFO: `/v3/projects/df958a944ae44a89a751341b9ac8cba6` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:46.345734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df958a944ae44a89a751341b9ac8cba6)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:46.348112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:46.506112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df958a944ae44a89a751341b9ac8cba6', 'request_id': 'req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:46.528757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df958a944ae44a89a751341b9ac8cba6', 'request_id': 'req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:46.610676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df958a944ae44a89a751341b9ac8cba6', 'request_id': 'req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:46.622818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] The token cache is being invalidate because project df958a944ae44a89a751341b9ac8cba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:46.623303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:46.624037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-eb414ed2-d096-4083-91a2-b0ad4f42f4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:46.625711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1190/2395] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:46 2022] DELETE /identity/v3/projects/df958a944ae44a89a751341b9ac8cba6 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:46.629991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:46.631455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:46.646558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e28c64-09ae-46c2-b5f7-5255c08e4569 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:46.725358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:46.726896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:46.727357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:46.727796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] PATH_INFO: `/v3/users/bd89a83221e6405591c917f96aeffd0b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:46.729794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd89a83221e6405591c917f96aeffd0b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:46.732468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:46.961789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd89a83221e6405591c917f96aeffd0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.065474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd89a83221e6405591c917f96aeffd0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.065474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:47.080247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd89a83221e6405591c917f96aeffd0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.081042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd89a83221e6405591c917f96aeffd0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.085676 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd89a83221e6405591c917f96aeffd0b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.107482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1206/2396] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:46 2022] DELETE /identity/v3/users/bd89a83221e6405591c917f96aeffd0b => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:47.127460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:47.127943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:47.128380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:47.164236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] MFA Rules not processed for user `4f791616148c4fa08c83c699b0a21077`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:47.165527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:47.246409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1191/2397] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:47.259719 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:47.261192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:47.337970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c1e93-d9fe-4cab-8148-d78742215aa6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:47.344418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949cda0-7de6-48e4-979d-f756c53d2d2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:47.345448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949cda0-7de6-48e4-979d-f756c53d2d2e service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:47.345738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949cda0-7de6-48e4-979d-f756c53d2d2e service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:47.348859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949cda0-7de6-48e4-979d-f756c53d2d2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:47.351029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6949cda0-7de6-48e4-979d-f756c53d2d2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:47.427770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949cda0-7de6-48e4-979d-f756c53d2d2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:47.430193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949cda0-7de6-48e4-979d-f756c53d2d2e service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:47.440110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1207/2398] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:47.635008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:47.637119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:47.702974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:47.743099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7991d5-7325-4cac-b60f-5c1a9135245a None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:47.748815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:47.749577 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:47.757208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:47.757208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] PATH_INFO: `/v3/projects/c452ddf22149433ba6d90bad3e05678f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:47.757208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c452ddf22149433ba6d90bad3e05678f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:47.757208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:47.876031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c452ddf22149433ba6d90bad3e05678f', 'request_id': 'req-00f2b82b-bedd-418b-8937-737239f336a5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.882479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c452ddf22149433ba6d90bad3e05678f', 'request_id': 'req-00f2b82b-bedd-418b-8937-737239f336a5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.942443 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c452ddf22149433ba6d90bad3e05678f', 'request_id': 'req-00f2b82b-bedd-418b-8937-737239f336a5'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.955716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] The token cache is being invalidate because project c452ddf22149433ba6d90bad3e05678f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:47.959703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.959703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:47.959703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1192/2399] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:47 2022] DELETE /identity/v3/projects/c452ddf22149433ba6d90bad3e05678f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:50.384107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8d1b-310b-4b6d-bdac-6dc2b06164fd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:50.384653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8d1b-310b-4b6d-bdac-6dc2b06164fd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:50.384653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8d1b-310b-4b6d-bdac-6dc2b06164fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:50.414922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-828f8d1b-310b-4b6d-bdac-6dc2b06164fd None None] MFA Rules not processed for user `0fe07f55f4d54059a193d4611e0a4f92`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:50.416168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-828f8d1b-310b-4b6d-bdac-6dc2b06164fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:50.509338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1208/2400] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:57.955388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:57.957429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:58.032122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2b82b-bedd-418b-8937-737239f336a5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:58.036786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:58.037407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:58.037677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:58.037939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] PATH_INFO: `/v3/users/1705522010054ab9895950bd0df202b3` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:58.039306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1705522010054ab9895950bd0df202b3)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:58.041496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:58.167797 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1705522010054ab9895950bd0df202b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:58.216961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1705522010054ab9895950bd0df202b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:58.222811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:58.229922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1705522010054ab9895950bd0df202b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:58.237332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1705522010054ab9895950bd0df202b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:58.237974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1705522010054ab9895950bd0df202b3'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:58.244317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1193/2401] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:35:57 2022] DELETE /identity/v3/users/1705522010054ab9895950bd0df202b3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:35:58.249734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:58.250256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:58.250375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:58.271168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:35:58.272380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:58.282425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:58.336189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1209/2402] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:35:58.347615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:58.349051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:58.413220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072f609-2bc3-45d4-b309-9f67f59d323d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:58.417055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c306a9-1d5c-4d89-b839-e363e7f64a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:58.417659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c306a9-1d5c-4d89-b839-e363e7f64a8f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:58.417998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c306a9-1d5c-4d89-b839-e363e7f64a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:58.418273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c306a9-1d5c-4d89-b839-e363e7f64a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:58.428009 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c306a9-1d5c-4d89-b839-e363e7f64a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:58.429598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c306a9-1d5c-4d89-b839-e363e7f64a8f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:58.440939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1194/2403] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:58.852214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:58.854064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:58.867651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2d5a5-4b70-4cd7-9b55-c504b17e5c62 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:58.874312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:58.875051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:58.875866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:58.876141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] PATH_INFO: `/v3/projects/d0f6186d2ab14414a1e0a41533a36e29` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:58.877461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f6186d2ab14414a1e0a41533a36e29)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:58.895515 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:59.001256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f6186d2ab14414a1e0a41533a36e29', 'request_id': 'req-61465409-1d56-4bd7-a1ba-c5e7efc97d22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:59.067949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f6186d2ab14414a1e0a41533a36e29', 'request_id': 'req-61465409-1d56-4bd7-a1ba-c5e7efc97d22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:59.074868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f6186d2ab14414a1e0a41533a36e29', 'request_id': 'req-61465409-1d56-4bd7-a1ba-c5e7efc97d22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:59.090815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] The token cache is being invalidate because project d0f6186d2ab14414a1e0a41533a36e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:35:59.091204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:59.091807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:35:59.093324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1210/2404] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:35:58 2022] DELETE /identity/v3/projects/d0f6186d2ab14414a1e0a41533a36e29 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:08.439489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=35,keystone:DELETE=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.084770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:02.772858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce685e-e0aa-4e2b-b4ca-faa18110cc7b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:02.773429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce685e-e0aa-4e2b-b4ca-faa18110cc7b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:02.773429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce685e-e0aa-4e2b-b4ca-faa18110cc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:02.792632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-5fce685e-e0aa-4e2b-b4ca-faa18110cc7b None None] MFA Rules not processed for user `f67323f672464220a38906660715a9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:02.793837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-5fce685e-e0aa-4e2b-b4ca-faa18110cc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:02.849877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1195/2405] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:37:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:04.195748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:04.197271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:04.255511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61465409-1d56-4bd7-a1ba-c5e7efc97d22 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:04.259093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:04.259690 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:04.259961 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:04.260229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:04.261925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:04.325301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:04.326787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:04.337251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1211/2406] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:12.803487 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce685e-e0aa-4e2b-b4ca-faa18110cc7b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:12.814449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce685e-e0aa-4e2b-b4ca-faa18110cc7b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:12.819572 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:12.820188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:12.820463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:12.820734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:12.823218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:12.911419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:12.912875 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:12.926653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1196/2407] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:14.332955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=50 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:14.813399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:14.814793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:14.826475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e511a-b805-4fd6-bb3e-d5816817fc95 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:14.830475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba82ff-1b05-4e42-8442-c1d086ba1d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:14.831075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba82ff-1b05-4e42-8442-c1d086ba1d56 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:14.831430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba82ff-1b05-4e42-8442-c1d086ba1d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:14.831608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba82ff-1b05-4e42-8442-c1d086ba1d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:14.834538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-92ba82ff-1b05-4e42-8442-c1d086ba1d56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:14.848953 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba82ff-1b05-4e42-8442-c1d086ba1d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:14.850609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba82ff-1b05-4e42-8442-c1d086ba1d56 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:14.874679 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1212/2408] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:21.887559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:21.889008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:21.902707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24ceb8-e847-4a58-aeee-db93cd4718d5 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:21.908331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:21.908924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:21.909189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:21.909454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:21.911676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:21.970711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:21.972800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:21.981725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1197/2409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:24.857363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:31.978048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:42.922440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb010a-7497-49f8-855d-fd44ef1cfeb5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:42.922440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb010a-7497-49f8-855d-fd44ef1cfeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:42.922440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb010a-7497-49f8-855d-fd44ef1cfeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:42.940282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-83bb010a-7497-49f8-855d-fd44ef1cfeb5 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:42.941523 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-83bb010a-7497-49f8-855d-fd44ef1cfeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:42.996626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1213/2410] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:37:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:43.037675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:43.039128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:43.109384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c2c4c-c3b4-49a5-804b-edea04f5b52b service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:43.115462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:43.116054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:43.116354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:43.116618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:43.124707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:43.126386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:43.135554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1198/2411] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:37:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:37:52.989947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=25 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:53.131937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=32 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:54.661616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb010a-7497-49f8-855d-fd44ef1cfeb5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:54.676903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb010a-7497-49f8-855d-fd44ef1cfeb5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:54.685367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:54.686754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:54.686754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:54.686754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:54.689159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:54.702793 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:54.704545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:54.717086 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1214/2412] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:57.123559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:57.133601 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7e6cc-774c-4406-8147-7b6e7f515847 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:57.137166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd202cf-7c02-43b3-812c-2ddb3ddddcf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:57.137754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd202cf-7c02-43b3-812c-2ddb3ddddcf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:57.138033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd202cf-7c02-43b3-812c-2ddb3ddddcf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:57.138625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd202cf-7c02-43b3-812c-2ddb3ddddcf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:57.140365 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1bd202cf-7c02-43b3-812c-2ddb3ddddcf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:57.151178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd202cf-7c02-43b3-812c-2ddb3ddddcf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:57.152638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd202cf-7c02-43b3-812c-2ddb3ddddcf3 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:57.160877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1199/2413] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:38:01.438624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:01.448740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c208c3-81f6-47e7-8d24-b74a43d621f6 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:01.452376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e41ec-32e1-413a-a4f3-67cb6670ebbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:01.452988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e41ec-32e1-413a-a4f3-67cb6670ebbc service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:01.453267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e41ec-32e1-413a-a4f3-67cb6670ebbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:01.453538 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e41ec-32e1-413a-a4f3-67cb6670ebbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:01.455526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e34e41ec-32e1-413a-a4f3-67cb6670ebbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:01.465461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e41ec-32e1-413a-a4f3-67cb6670ebbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:01.466936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e41ec-32e1-413a-a4f3-67cb6670ebbc service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:01.474860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1215/2414] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:01.856414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7f1d3-f681-4a74-870b-b3eaf54dcbd3 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:01.856897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7f1d3-f681-4a74-870b-b3eaf54dcbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:01.856993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7f1d3-f681-4a74-870b-b3eaf54dcbd3 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:01.858301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1200/2415] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:38:01.864364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0663a3-47bc-43e6-87d6-4420e2380dcc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:01.864699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0663a3-47bc-43e6-87d6-4420e2380dcc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:01.864975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0663a3-47bc-43e6-87d6-4420e2380dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:01.885808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9f0663a3-47bc-43e6-87d6-4420e2380dcc None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:01.887065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9f0663a3-47bc-43e6-87d6-4420e2380dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:01.896492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1216/2416] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:38:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:01.907856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7f1d3-f681-4a74-870b-b3eaf54dcbd3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:01.909726 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-08b7f1d3-f681-4a74-870b-b3eaf54dcbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:01.975031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7f1d3-f681-4a74-870b-b3eaf54dcbd3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:01.978729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60d7f5-bf2d-4289-90e1-cb074a39975a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:01.979410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60d7f5-bf2d-4289-90e1-cb074a39975a service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:01.979685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60d7f5-bf2d-4289-90e1-cb074a39975a service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:01.979952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60d7f5-bf2d-4289-90e1-cb074a39975a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:01.989786 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60d7f5-bf2d-4289-90e1-cb074a39975a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:01.991238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60d7f5-bf2d-4289-90e1-cb074a39975a service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:02.002437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1201/2417] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:02.204600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834a67-e1b0-4480-b2d1-fd161f2c0f17 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:02.205133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834a67-e1b0-4480-b2d1-fd161f2c0f17 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:02.205400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0834a67-e1b0-4480-b2d1-fd161f2c0f17 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:02.206785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1217/2418] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:38:02.211321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823aaca-6cd3-492d-a12c-3c826d737689 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:02.215339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823aaca-6cd3-492d-a12c-3c826d737689 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:02.215339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823aaca-6cd3-492d-a12c-3c826d737689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:02.232464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b823aaca-6cd3-492d-a12c-3c826d737689 None None] MFA Rules not processed for user `e8839af749e44fac8a7a311b61463c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:02.233680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b823aaca-6cd3-492d-a12c-3c826d737689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:02.240213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1202/2419] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:38:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:02.253184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0834a67-e1b0-4480-b2d1-fd161f2c0f17 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:02.264117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0834a67-e1b0-4480-b2d1-fd161f2c0f17 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:02.267678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa6a51-f857-47ab-aba5-1a916378fd05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:02.268259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa6a51-f857-47ab-aba5-1a916378fd05 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:02.268540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa6a51-f857-47ab-aba5-1a916378fd05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:02.268806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa6a51-f857-47ab-aba5-1a916378fd05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:02.275928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa6a51-f857-47ab-aba5-1a916378fd05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:02.277324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa6a51-f857-47ab-aba5-1a916378fd05 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:02.285001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1218/2420] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:02.367128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18823-8b29-4a3d-91f7-2ddc7dcb5d99 None None] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:02.367411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18823-8b29-4a3d-91f7-2ddc7dcb5d99 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:02.367677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18823-8b29-4a3d-91f7-2ddc7dcb5d99 None None] PATH_INFO: `/` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:02.369023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1203/2421] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 02:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:38:02.374780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:02.375066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:02.375340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:02.394617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] MFA Rules not processed for user `c4cfb26c57514e2cbaa73cb036118dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:02.395818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:02.405073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1219/2422] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 02:38:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:02.415278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18823-8b29-4a3d-91f7-2ddc7dcb5d99 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:02.426707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18823-8b29-4a3d-91f7-2ddc7dcb5d99 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:02.430276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0285626-100f-492b-ae2d-b1ed4e3976b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:02.430888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285626-100f-492b-ae2d-b1ed4e3976b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:02.431190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285626-100f-492b-ae2d-b1ed4e3976b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:02.431454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285626-100f-492b-ae2d-b1ed4e3976b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:02.441923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0285626-100f-492b-ae2d-b1ed4e3976b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:02.443362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0285626-100f-492b-ae2d-b1ed4e3976b0 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:02.453887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1204/2423] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:02.517552 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:02.519061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:02.587421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bade3-40fb-4d34-a16d-ee236964d292 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:02.593268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:02.593858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:02.594134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:02.594429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:02.596154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:02.608356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:02.609985 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:02.619694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1220/2424] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:07.686398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:07.686885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:07.687202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:07.708275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:07.709476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:07.717251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1205/2425] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:38:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:07.730556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:07.741328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5020-378c-4d4a-9a40-17096ce27b66 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:07.746657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:07.747334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:07.747616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:07.747891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:07.756812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:07.758756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:07.767359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1221/2426] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:38:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:38:17.703492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=44 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:17.764175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=62 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:36.362814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:36.364456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:36.433647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa7623-11c0-4fed-8809-d6d1a3bb1053 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:36.438460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:36.439079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:36.439358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:36.439629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] PATH_INFO: `/v3/users/f67323f672464220a38906660715a9d6` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:36.440781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67323f672464220a38906660715a9d6)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:36.442774 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:36.591492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67323f672464220a38906660715a9d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:36.636294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67323f672464220a38906660715a9d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:36.649655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67323f672464220a38906660715a9d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:36.657762 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67323f672464220a38906660715a9d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:36.658744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67323f672464220a38906660715a9d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:36.664706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1206/2427] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:38:36 2022] DELETE /identity/v3/users/f67323f672464220a38906660715a9d6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:38:36.927274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:36.938647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885680d-17fa-4935-9c05-b7abd20c6e6b service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:36.995479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:36.996104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:36.996380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:36.996643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] PATH_INFO: `/v3/projects/5ec7a76094c54d6c8e2517e254f44c16` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:36.998410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec7a76094c54d6c8e2517e254f44c16)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:37.000321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:37.092084 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec7a76094c54d6c8e2517e254f44c16', 'request_id': 'req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:37.159572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec7a76094c54d6c8e2517e254f44c16', 'request_id': 'req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:37.165955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec7a76094c54d6c8e2517e254f44c16', 'request_id': 'req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:37.176300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] The token cache is being invalidate because project 5ec7a76094c54d6c8e2517e254f44c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:38:37.176613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:37.177296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-cb5e1238-0858-42fd-881e-b0b1b6d61aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:38:37.178808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1222/2428] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:38:36 2022] DELETE /identity/v3/projects/5ec7a76094c54d6c8e2517e254f44c16 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:38:38.654978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:38.656504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:38.729544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee5908-ed83-434d-b47f-eaa8f293eb1f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:38.742344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:38.794768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:38.795372 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:38.795651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:38.795923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:38.797662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:38.812625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:38.812625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:38.822586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1207/2429] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:42.162375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:42.162871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:42.162968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:42.186311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:38:42.186311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:42.242559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1223/2430] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 02:38:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:38:42.256027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:42.257468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:42.324122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaa840-3d3c-4af1-b50b-508aa0c10f0c service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:42.329879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62fd04-e1cf-4e78-91e0-e36e77c5f617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:42.330520 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62fd04-e1cf-4e78-91e0-e36e77c5f617 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:42.330800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62fd04-e1cf-4e78-91e0-e36e77c5f617 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:42.331070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62fd04-e1cf-4e78-91e0-e36e77c5f617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:42.341182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62fd04-e1cf-4e78-91e0-e36e77c5f617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:42.342821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62fd04-e1cf-4e78-91e0-e36e77c5f617 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:42.351563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1208/2431] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:38:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:38:52.235610 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:52.347838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=58 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:15.214882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:15.217885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:15.232294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4409d4b-2ad2-41d0-a21f-22e61dc23e7a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:15.238231 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:15.238847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:15.239124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:15.239395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:15.241080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:15.299447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:15.301027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:15.310334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1224/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:39:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:39:25.305964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=28 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:46.508563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698cc6d-20d4-4906-9bfe-f38ef4b5a106 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:46.509158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698cc6d-20d4-4906-9bfe-f38ef4b5a106 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:46.509158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698cc6d-20d4-4906-9bfe-f38ef4b5a106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:46.528416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c698cc6d-20d4-4906-9bfe-f38ef4b5a106 None None] MFA Rules not processed for user `f6fb96d9442147e0a2201fe890d6f955`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:39:46.529641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c698cc6d-20d4-4906-9bfe-f38ef4b5a106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:46.537497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1209/2433] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:39:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:39:51.606229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:51.618885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a7873-9855-4600-b8a5-e20f7a58e8ea service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:51.625952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209bb2b-990b-4b96-987f-3406c3682f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:51.626890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bb2b-990b-4b96-987f-3406c3682f07 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:51.627175 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bb2b-990b-4b96-987f-3406c3682f07 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:51.627627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bb2b-990b-4b96-987f-3406c3682f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:51.632224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e209bb2b-990b-4b96-987f-3406c3682f07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:51.646022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209bb2b-990b-4b96-987f-3406c3682f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:51.647642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209bb2b-990b-4b96-987f-3406c3682f07 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:51.658606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1225/2434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:39:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:39:56.523116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:01.653701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:03.579934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698cc6d-20d4-4906-9bfe-f38ef4b5a106 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:03.593597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698cc6d-20d4-4906-9bfe-f38ef4b5a106 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:03.599468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c48ed-3ff7-4cc3-81fa-f2d58bde3b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:03.600052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c48ed-3ff7-4cc3-81fa-f2d58bde3b62 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:03.600325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c48ed-3ff7-4cc3-81fa-f2d58bde3b62 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:03.600593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c48ed-3ff7-4cc3-81fa-f2d58bde3b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:03.602295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-689c48ed-3ff7-4cc3-81fa-f2d58bde3b62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:03.660002 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c48ed-3ff7-4cc3-81fa-f2d58bde3b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:03.661588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c48ed-3ff7-4cc3-81fa-f2d58bde3b62 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:03.670377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1210/2435] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:12.820368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557cda8-6e1c-4d80-8d17-3f5fb26c5431 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:12.820893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557cda8-6e1c-4d80-8d17-3f5fb26c5431 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:12.820893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557cda8-6e1c-4d80-8d17-3f5fb26c5431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:12.840220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2557cda8-6e1c-4d80-8d17-3f5fb26c5431 None None] MFA Rules not processed for user `7fd9eb21aa634650a7022b7b4ca145fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:40:12.841416 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2557cda8-6e1c-4d80-8d17-3f5fb26c5431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:12.896457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1226/2436] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:40:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:40:13.462531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b314d-1128-46dc-8022-f59763725f52 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:13.463033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b314d-1128-46dc-8022-f59763725f52 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:13.463144 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b314d-1128-46dc-8022-f59763725f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:13.483879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-542b314d-1128-46dc-8022-f59763725f52 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:40:13.485064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-542b314d-1128-46dc-8022-f59763725f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:13.493060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1211/2437] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 02:40:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:40:13.653436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557cda8-6e1c-4d80-8d17-3f5fb26c5431 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:13.666539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557cda8-6e1c-4d80-8d17-3f5fb26c5431 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:13.671967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:13.672623 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:13.673011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:13.673328 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:13.682665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:13.684224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:13.693379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1227/2438] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:40:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:40:14.791611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b314d-1128-46dc-8022-f59763725f52 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:14.801892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-542b314d-1128-46dc-8022-f59763725f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:14.851042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b314d-1128-46dc-8022-f59763725f52 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:14.855060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:14.855670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:14.855946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:14.856219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:14.858039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:14.923535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:14.925219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:14.935223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1212/2439] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:18.954036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:18.956103 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:19.108830 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c329ed4-ee0f-4a19-b1fc-d303c068ee40 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:19.114056 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:19.114688 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:19.114964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:19.115266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:19.117074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:19.130730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:19.132335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:19.143774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1228/2440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:22.848907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:22.850400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:22.899501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:22.929749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdf8cd-d66a-4501-8023-ca1fd21fba08 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:22.934923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:22.935546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:22.935825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:22.936097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] PATH_INFO: `/v3/users/747b60cf32a14b93bee399fce7a9b54b` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:22.937220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] RBAC: Authorizing `identity:delete_user(user_id=747b60cf32a14b93bee399fce7a9b54b)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:22.939688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:23.124986 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747b60cf32a14b93bee399fce7a9b54b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.188021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747b60cf32a14b93bee399fce7a9b54b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.201398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747b60cf32a14b93bee399fce7a9b54b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.209462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747b60cf32a14b93bee399fce7a9b54b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.210117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747b60cf32a14b93bee399fce7a9b54b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.216112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1213/2441] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:40:22 2022] DELETE /identity/v3/users/747b60cf32a14b93bee399fce7a9b54b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:40:23.494772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:23.512418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc48f1-2985-4887-b32a-837ece5cbe82 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:23.547562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:23.577702 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:23.582663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:23.582663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:23.582663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] PATH_INFO: `/v3/projects/31bdca178d874d499b1492305ad45a9c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:23.582663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31bdca178d874d499b1492305ad45a9c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:23.582663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:23.715429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31bdca178d874d499b1492305ad45a9c', 'request_id': 'req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.801215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31bdca178d874d499b1492305ad45a9c', 'request_id': 'req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.807946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31bdca178d874d499b1492305ad45a9c', 'request_id': 'req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.819650 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] The token cache is being invalidate because project 31bdca178d874d499b1492305ad45a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:40:23.819942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.820590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:23.822296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1229/2442] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:40:23 2022] DELETE /identity/v3/projects/31bdca178d874d499b1492305ad45a9c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:40:23.826281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:23.827710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:23.843297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fa197-32c2-46b9-8386-a936a95f043c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:23.900884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:23.901536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:23.901811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:23.902079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] PATH_INFO: `/v3/users/44630f5248e04ed99c59b42397a6c273` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:23.903214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] RBAC: Authorizing `identity:delete_user(user_id=44630f5248e04ed99c59b42397a6c273)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:23.905143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:24.105822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44630f5248e04ed99c59b42397a6c273'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.160971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44630f5248e04ed99c59b42397a6c273'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.162243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:24.173837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44630f5248e04ed99c59b42397a6c273'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.181354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44630f5248e04ed99c59b42397a6c273'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.182058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44630f5248e04ed99c59b42397a6c273'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.188153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1214/2443] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:40:23 2022] DELETE /identity/v3/users/44630f5248e04ed99c59b42397a6c273 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:40:24.465567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:24.467032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:24.480663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cddf68-8fe3-48a1-a3ed-0e8c7172af47 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:24.538503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:24.539142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:24.539397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:24.539668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] PATH_INFO: `/v3/projects/d6ff6b4542fb4445865c6a9cd7923b73` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:24.540788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6ff6b4542fb4445865c6a9cd7923b73)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:24.542723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:24.667973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6ff6b4542fb4445865c6a9cd7923b73', 'request_id': 'req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.742222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6ff6b4542fb4445865c6a9cd7923b73', 'request_id': 'req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.749039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6ff6b4542fb4445865c6a9cd7923b73', 'request_id': 'req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.754695 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:24.762531 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] The token cache is being invalidate because project d6ff6b4542fb4445865c6a9cd7923b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:40:24.762817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.763458 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:40:24.764975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1230/2444] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:40:24 2022] DELETE /identity/v3/projects/d6ff6b4542fb4445865c6a9cd7923b73 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:40:34.755104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=11,keystone:DELETE=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:34.976645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:41.748358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:41.749829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:41.843936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24162d-d4d8-419d-9e25-abfa59d1686d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:41.903717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:41.904358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:41.904636 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:41.904909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:41.906660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:41.919099 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:41.920656 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:41.931619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1215/2445] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:43.047301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:43.048758 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:43.130839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45f610-f588-4d7e-bacb-9b39e8bf8e93 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:43.134467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:43.135047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:43.135325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:43.135586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:43.137313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:43.147330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:43.148795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:43.156866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1231/2446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:45.117941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:45.119488 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:45.190317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=100 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:45.225051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bee5d0-e818-4a82-90b6-b5bd614d0593 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:45.230698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:45.231296 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:45.231576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:45.231846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:45.234059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:45.294727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:45.296303 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:45.305192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1216/2447] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:46.964600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:46.979137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa620-9616-469c-a353-67b36b5cbef1 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:46.992019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:46.992975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:46.993370 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:46.993647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:46.995551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:47.060890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:47.063126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:47.074044 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1232/2448] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:55.301546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=35 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:57.069209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=63 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:15.619669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:15.621108 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:15.633027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da17881-99a1-46f4-88a3-aca9cd215d2d service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:15.636648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:15.637237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:15.637539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:15.637809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:15.639504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:15.701979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:15.703710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:15.713859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1217/2449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:19.166773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:19.168197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:19.182448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ef14a-e9ad-4edc-bb42-a3cb3cd555b7 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:19.187443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:19.188027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:19.188332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:19.188571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:19.190243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:19.202315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:19.204209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:19.216108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1233/2450] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:19.350089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:19.351604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:19.363612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee67ab-55f0-48c9-bdec-af98e0ecf22b service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:19.368917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:19.369531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:19.369831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:19.370073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] PATH_INFO: `/v3/users/f6fb96d9442147e0a2201fe890d6f955` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:19.371251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6fb96d9442147e0a2201fe890d6f955)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:19.373149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:19.595116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6fb96d9442147e0a2201fe890d6f955'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:19.656135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6fb96d9442147e0a2201fe890d6f955'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:19.668689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6fb96d9442147e0a2201fe890d6f955'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:19.675471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6fb96d9442147e0a2201fe890d6f955'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:19.676124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-46c19d70-f258-46d4-b359-7cadafd985eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6fb96d9442147e0a2201fe890d6f955'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:19.682535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1218/2451] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:19 2022] DELETE /identity/v3/users/f6fb96d9442147e0a2201fe890d6f955 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:41:19.910734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:19.928239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a45b3-799d-4118-9b9e-4a7ba1aafd56 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:19.987477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:19.988089 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:19.988361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:19.988626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] PATH_INFO: `/v3/projects/f1f194384993444da5c7b7f30e42746d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:19.989725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f194384993444da5c7b7f30e42746d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:19.991627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:20.097818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f194384993444da5c7b7f30e42746d', 'request_id': 'req-837cf501-5c70-49a4-a3fe-5de78ad1e296'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:20.166630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f194384993444da5c7b7f30e42746d', 'request_id': 'req-837cf501-5c70-49a4-a3fe-5de78ad1e296'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:20.173069 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f194384993444da5c7b7f30e42746d', 'request_id': 'req-837cf501-5c70-49a4-a3fe-5de78ad1e296'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:20.183657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] The token cache is being invalidate because project f1f194384993444da5c7b7f30e42746d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:20.183971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:20.184607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-837cf501-5c70-49a4-a3fe-5de78ad1e296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:20.186110 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1234/2452] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:19 2022] DELETE /identity/v3/projects/f1f194384993444da5c7b7f30e42746d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:20.191036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3e639-1f55-4faa-93c3-9c207def135d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.191323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3e639-1f55-4faa-93c3-9c207def135d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.191622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3e639-1f55-4faa-93c3-9c207def135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.249638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-d0e3e639-1f55-4faa-93c3-9c207def135d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:20.250989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d0e3e639-1f55-4faa-93c3-9c207def135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:20.282596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:20.312208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1219/2453] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:20.319963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fc655-426d-4742-bb85-2f06d8b6e696 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.320252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fc655-426d-4742-bb85-2f06d8b6e696 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.320529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fc655-426d-4742-bb85-2f06d8b6e696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.340947 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-204fc655-426d-4742-bb85-2f06d8b6e696 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:20.342157 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-204fc655-426d-4742-bb85-2f06d8b6e696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:20.350744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1235/2454] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:20.355620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3e639-1f55-4faa-93c3-9c207def135d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:20.365675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3e639-1f55-4faa-93c3-9c207def135d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:20.370403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:20.370996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.371275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.371549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.372224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:41:20.372524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:20.374115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:20.385962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1220/2455] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:41:20.389700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fc655-426d-4742-bb85-2f06d8b6e696 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:20.400189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fc655-426d-4742-bb85-2f06d8b6e696 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:20.405398 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:20.406029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.406373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.406728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.407369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:20.409285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:20.503508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1236/2456] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:41:20 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:41:20.507692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:20.519651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf882a92-65fb-42a5-89c0-25a991ba7a56 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:20.575266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:20.575990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.576199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.576687 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.577523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:20.580280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:20.663465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1221/2457] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:41:20 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:20.668996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:20.679438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a4e1b-3ec1-4a7e-b043-97ccd0a76622 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:20.684176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:20.684844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.685123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.685391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.686027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:20.687646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:20.698298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1237/2458] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:20.702242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:20.713265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52cded-1991-42e9-9321-39b031460042 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:20.717944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04fe54-dff7-4dc5-b572-3c821a177235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:20.718568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04fe54-dff7-4dc5-b572-3c821a177235 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.718848 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04fe54-dff7-4dc5-b572-3c821a177235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.719116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04fe54-dff7-4dc5-b572-3c821a177235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.719742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04fe54-dff7-4dc5-b572-3c821a177235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:20.721320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04fe54-dff7-4dc5-b572-3c821a177235 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:20.731766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1222/2459] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:20.735902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:20.753088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a844-e3cd-4126-9e15-62c3604cd2dc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:20.753088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:20.753088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.753088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.753570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] PATH_INFO: `/v3/projects/2818e0e2e04d47998507aa18da59341e/users/94b695914ada4d339eeebe8cdb1b3f63/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.767122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2818e0e2e04d47998507aa18da59341e, user_id=94b695914ada4d339eeebe8cdb1b3f63, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:20.769756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:20.829160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1238/2460] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:20 2022] PUT /identity/v3/projects/2818e0e2e04d47998507aa18da59341e/users/94b695914ada4d339eeebe8cdb1b3f63/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:20.839338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fcdc1-cd52-4b73-956d-13ce0b1dd05b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:20.839626 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fcdc1-cd52-4b73-956d-13ce0b1dd05b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:20.839899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fcdc1-cd52-4b73-956d-13ce0b1dd05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:20.866699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-941fcdc1-cd52-4b73-956d-13ce0b1dd05b None None] MFA Rules not processed for user `94b695914ada4d339eeebe8cdb1b3f63`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:20.868462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-941fcdc1-cd52-4b73-956d-13ce0b1dd05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:20.932617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1223/2461] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:20.937077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:20.951176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:21.000118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626820e-d18e-4566-aa2d-24c9af883681 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.005436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.005977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.006256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.006557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.007192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.009874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:21.087862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1239/2462] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:41:20 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:21.091853 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fcdc1-cd52-4b73-956d-13ce0b1dd05b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:21.102852 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fcdc1-cd52-4b73-956d-13ce0b1dd05b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.131111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:21.158746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.159377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.159653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.159923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.160549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.162460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:21.252042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1224/2463] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:41:21 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:21.256408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:21.266990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef3ab1-3da7-414d-be3c-32123f9d0185 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.271660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.272246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.272525 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.272795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.273430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.275142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:21.285530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1240/2464] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:21.289532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:21.300031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e1570-db7e-40c4-8e51-c3d7bbc785d2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.304630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.305269 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.306167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.306469 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] PATH_INFO: `/v3/projects/7b0a4d72c5d94fc7a6cde493cb004416/users/4209c7f1f5cf4c71b48c8791f9354f55/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.321117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0a4d72c5d94fc7a6cde493cb004416, user_id=4209c7f1f5cf4c71b48c8791f9354f55, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.323792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:21.394546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1225/2465] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:21 2022] PUT /identity/v3/projects/7b0a4d72c5d94fc7a6cde493cb004416/users/4209c7f1f5cf4c71b48c8791f9354f55/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:21.398493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:21.408444 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd116bc1-c031-4602-9896-b1b53a2bf8d4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.464763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93992bf0-000a-4031-8b61-89619a42e380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.465361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93992bf0-000a-4031-8b61-89619a42e380 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.465639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93992bf0-000a-4031-8b61-89619a42e380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.465911 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93992bf0-000a-4031-8b61-89619a42e380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.466570 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93992bf0-000a-4031-8b61-89619a42e380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.468140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93992bf0-000a-4031-8b61-89619a42e380 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:21.478430 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1241/2466] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:21.482734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:21.493718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869021-facb-4e7a-94c5-0b9ef4184c84 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.498437 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.499080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.499360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.499633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] PATH_INFO: `/v3/projects/7b0a4d72c5d94fc7a6cde493cb004416/users/4209c7f1f5cf4c71b48c8791f9354f55/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.501563 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0a4d72c5d94fc7a6cde493cb004416, user_id=4209c7f1f5cf4c71b48c8791f9354f55, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.504274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:21.514383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1226/2467] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:21 2022] PUT /identity/v3/projects/7b0a4d72c5d94fc7a6cde493cb004416/users/4209c7f1f5cf4c71b48c8791f9354f55/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:21.524863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7094b-5d5a-4269-aa6e-cbf01b9afd56 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.525151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7094b-5d5a-4269-aa6e-cbf01b9afd56 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.525426 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7094b-5d5a-4269-aa6e-cbf01b9afd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.553467 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-61d7094b-5d5a-4269-aa6e-cbf01b9afd56 None None] MFA Rules not processed for user `4209c7f1f5cf4c71b48c8791f9354f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:21.554759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-61d7094b-5d5a-4269-aa6e-cbf01b9afd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:21.623663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1242/2468] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:21.634900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:21.636685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:21.715867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06938-280e-499c-b26b-cf235c8083f7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.719540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6489382-5441-41af-8478-5d6a1c891ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.720134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6489382-5441-41af-8478-5d6a1c891ffe service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.720409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6489382-5441-41af-8478-5d6a1c891ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.720676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6489382-5441-41af-8478-5d6a1c891ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.730246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6489382-5441-41af-8478-5d6a1c891ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.731702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6489382-5441-41af-8478-5d6a1c891ffe service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:21.740881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1227/2469] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:21.769019 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7094b-5d5a-4269-aa6e-cbf01b9afd56 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:21.779494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:21.832986 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7094b-5d5a-4269-aa6e-cbf01b9afd56 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:21.838375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:21.839000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:21.839359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:21.839591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] PATH_INFO: `/v3/users/94b695914ada4d339eeebe8cdb1b3f63` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:21.840733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b695914ada4d339eeebe8cdb1b3f63)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:21.842680 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:22.054371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b695914ada4d339eeebe8cdb1b3f63'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.137562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b695914ada4d339eeebe8cdb1b3f63'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.150512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b695914ada4d339eeebe8cdb1b3f63'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.151153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b695914ada4d339eeebe8cdb1b3f63'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.151761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b695914ada4d339eeebe8cdb1b3f63'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.164045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1243/2470] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:21 2022] DELETE /identity/v3/users/94b695914ada4d339eeebe8cdb1b3f63 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:22.169187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:22.169474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:22.169747 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:22.189244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:22.190482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:22.247696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1228/2471] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:22.257605 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:22.259176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:22.316371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258a4bc-a7eb-4889-8a4d-679f9725cb24 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:22.319970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:22.320557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:22.320847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:22.321300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:22.330376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:22.331806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:22.341817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1244/2472] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:22.721689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:22.723180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:22.735508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b26ce0-a4d7-44cd-874d-0ebba1fa47c9 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:22.740197 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:22.740826 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:22.741102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:22.741369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] PATH_INFO: `/v3/projects/2818e0e2e04d47998507aa18da59341e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:22.742562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2818e0e2e04d47998507aa18da59341e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:22.744501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:22.746285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:22.857913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2818e0e2e04d47998507aa18da59341e', 'request_id': 'req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.863996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2818e0e2e04d47998507aa18da59341e', 'request_id': 'req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.947815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2818e0e2e04d47998507aa18da59341e', 'request_id': 'req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.958995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] The token cache is being invalidate because project 2818e0e2e04d47998507aa18da59341e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:22.959281 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.959896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:22.961374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1229/2473] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:22 2022] DELETE /identity/v3/projects/2818e0e2e04d47998507aa18da59341e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:22.965739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:22.975528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518768db-3f4b-4348-94f5-defca3d71836 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:23.006025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:23.034701 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:23.035319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:23.035593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:23.035856 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] PATH_INFO: `/v3/users/4209c7f1f5cf4c71b48c8791f9354f55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:23.036977 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4209c7f1f5cf4c71b48c8791f9354f55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:23.038964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:23.224013 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4209c7f1f5cf4c71b48c8791f9354f55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:23.376357 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4209c7f1f5cf4c71b48c8791f9354f55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:23.389000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4209c7f1f5cf4c71b48c8791f9354f55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:23.389635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4209c7f1f5cf4c71b48c8791f9354f55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:23.390262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6881f62c-1747-472c-8a6e-525d4f45215c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4209c7f1f5cf4c71b48c8791f9354f55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:23.403142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1245/2474] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:22 2022] DELETE /identity/v3/users/4209c7f1f5cf4c71b48c8791f9354f55 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:23.808651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:23.812302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:23.826937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0327c-4fe5-4df0-a9c9-0396c99cd91f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:23.885538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:23.886176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:23.886492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:23.886769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] PATH_INFO: `/v3/projects/7b0a4d72c5d94fc7a6cde493cb004416` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:23.887924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0a4d72c5d94fc7a6cde493cb004416)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:23.889834 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:23.983415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0a4d72c5d94fc7a6cde493cb004416', 'request_id': 'req-5417b551-4332-41f6-9d3f-6ff7b6018776'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:23.989620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0a4d72c5d94fc7a6cde493cb004416', 'request_id': 'req-5417b551-4332-41f6-9d3f-6ff7b6018776'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:24.051823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0a4d72c5d94fc7a6cde493cb004416', 'request_id': 'req-5417b551-4332-41f6-9d3f-6ff7b6018776'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:24.062928 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] The token cache is being invalidate because project 7b0a4d72c5d94fc7a6cde493cb004416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:24.063217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:24.063865 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-5417b551-4332-41f6-9d3f-6ff7b6018776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:24.065335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1230/2475] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:23 2022] DELETE /identity/v3/projects/7b0a4d72c5d94fc7a6cde493cb004416 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:24.070242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec480d1-0b41-412b-853e-b5d02d06bc8c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.070562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec480d1-0b41-412b-853e-b5d02d06bc8c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.071514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec480d1-0b41-412b-853e-b5d02d06bc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.091246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-eec480d1-0b41-412b-853e-b5d02d06bc8c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:24.092449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eec480d1-0b41-412b-853e-b5d02d06bc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:24.151644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1246/2476] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:24.160948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a62e37-465a-4c1b-9173-9c5fc860340c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.161236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a62e37-465a-4c1b-9173-9c5fc860340c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.161508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a62e37-465a-4c1b-9173-9c5fc860340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.182279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-38a62e37-465a-4c1b-9173-9c5fc860340c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:24.183505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-38a62e37-465a-4c1b-9173-9c5fc860340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:24.191752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1231/2477] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:24.196067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec480d1-0b41-412b-853e-b5d02d06bc8c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.205945 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec480d1-0b41-412b-853e-b5d02d06bc8c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:24.210509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:24.211118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.211496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.211958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.212637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:41:24.212948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:24.214583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:24.225995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1247/2478] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:41:24.230764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a62e37-465a-4c1b-9173-9c5fc860340c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.240515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a62e37-465a-4c1b-9173-9c5fc860340c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:24.245979 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:24.246608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.246867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.247132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.247998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:24.249878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:24.261531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:24.315975 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1232/2479] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:41:24 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:41:24.320190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.330501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c74797-e758-4b44-8a01-d2a7a12a7e21 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:24.365126 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:24.389001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:24.389597 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.389870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.390143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.390805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:24.392723 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:24.457983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1248/2480] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:41:24 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:24.462445 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.472897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c768c2d-274e-4ca8-9063-1559bc7d0c02 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:24.478143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:24.478902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.479226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.479586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.480386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:24.482104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:24.492040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1233/2481] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:24.496445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.506483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be5e0-822c-4517-b3bb-ca1d1342a953 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:24.511316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bf931-0bc5-4441-96f9-b41981598443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:24.511895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bf931-0bc5-4441-96f9-b41981598443 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.512179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bf931-0bc5-4441-96f9-b41981598443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.512449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bf931-0bc5-4441-96f9-b41981598443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.513080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bf931-0bc5-4441-96f9-b41981598443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:24.514692 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bf931-0bc5-4441-96f9-b41981598443 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:24.524984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1249/2482] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:24.528949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.538754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a9e0-c728-481c-a1fb-52c89cf94719 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:24.544014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:24.544642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.544918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.545187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] PATH_INFO: `/v3/projects/12ed26f0b8b14e7ea9d710986bf0b022/users/03157413068a467cb049c3f9749270e9/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.559525 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ed26f0b8b14e7ea9d710986bf0b022, user_id=03157413068a467cb049c3f9749270e9, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:24.562168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:24.652671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1234/2483] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:24 2022] PUT /identity/v3/projects/12ed26f0b8b14e7ea9d710986bf0b022/users/03157413068a467cb049c3f9749270e9/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:24.668669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5049c6-cbd1-4e6b-ba10-1a5f2a221153 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.668952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5049c6-cbd1-4e6b-ba10-1a5f2a221153 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.669223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5049c6-cbd1-4e6b-ba10-1a5f2a221153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.696497 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-da5049c6-cbd1-4e6b-ba10-1a5f2a221153 None None] MFA Rules not processed for user `03157413068a467cb049c3f9749270e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:24.697747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-da5049c6-cbd1-4e6b-ba10-1a5f2a221153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:24.760689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1250/2484] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:24.765243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.826026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fcec-98fd-4db5-bc0e-33592d7e1abb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:24.831153 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:24.831755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:24.832039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:24.832313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:24.833000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:24.834982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:24.927697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1235/2485] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:41:24 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:24.931965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5049c6-cbd1-4e6b-ba10-1a5f2a221153 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:24.943068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5049c6-cbd1-4e6b-ba10-1a5f2a221153 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:25.001542 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:25.002142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.002445 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.002717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.003331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:25.005247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:25.107475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1251/2486] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:41:24 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:25.111484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:25.121840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53774-13c4-4cd9-8d60-11bd14fe5810 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:25.126587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:25.127182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.127458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.127730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.128362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:25.129948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:25.141100 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1236/2487] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:25.145112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:25.155736 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344ecd0-7336-417b-bffa-949a2d0579e2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:25.160265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:25.160895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.161168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.161436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] PATH_INFO: `/v3/projects/a958764554444d23803dc51a1ca680f4/users/6afc104338a54e15889d4d73e52780ec/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.176173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a958764554444d23803dc51a1ca680f4, user_id=6afc104338a54e15889d4d73e52780ec, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:25.178759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:25.251674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1252/2488] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:25 2022] PUT /identity/v3/projects/a958764554444d23803dc51a1ca680f4/users/6afc104338a54e15889d4d73e52780ec/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:25.255902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:25.266531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab280df-0359-4a42-bded-2f7430b492ba admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:25.324202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a2eb4-fb19-4a11-aaab-c2018337a4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:25.324794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a2eb4-fb19-4a11-aaab-c2018337a4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.325074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a2eb4-fb19-4a11-aaab-c2018337a4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.325339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a2eb4-fb19-4a11-aaab-c2018337a4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.325964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a2eb4-fb19-4a11-aaab-c2018337a4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:25.327564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a2eb4-fb19-4a11-aaab-c2018337a4ed admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:25.337845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1237/2489] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:25.342096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:25.352734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2eded-2be6-49c8-a279-94d97f18190e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:25.357307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:25.357931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.358203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.358505 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] PATH_INFO: `/v3/projects/a958764554444d23803dc51a1ca680f4/users/6afc104338a54e15889d4d73e52780ec/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.360380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a958764554444d23803dc51a1ca680f4, user_id=6afc104338a54e15889d4d73e52780ec, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:25.363002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:25.369332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:25.374428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1253/2490] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:25 2022] PUT /identity/v3/projects/a958764554444d23803dc51a1ca680f4/users/6afc104338a54e15889d4d73e52780ec/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:25.391109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977614b-7252-4afb-baf0-778849385ed0 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.391392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977614b-7252-4afb-baf0-778849385ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.391665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977614b-7252-4afb-baf0-778849385ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.419078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8977614b-7252-4afb-baf0-778849385ed0 None None] MFA Rules not processed for user `6afc104338a54e15889d4d73e52780ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:25.420264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8977614b-7252-4afb-baf0-778849385ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:25.433692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:25.488812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1238/2491] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:25.500839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:25.502754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:25.559712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a28ae-3d13-41b5-a40d-9deb63a83eac admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:25.563433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:25.564023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.564303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.564567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.574211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:25.575667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:25.585200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1254/2492] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:25.822227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977614b-7252-4afb-baf0-778849385ed0 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:25.831961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977614b-7252-4afb-baf0-778849385ed0 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:25.843743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86642ea4-22e6-4c44-9bc6-3bd052e05dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:25.843743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86642ea4-22e6-4c44-9bc6-3bd052e05dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:25.843743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86642ea4-22e6-4c44-9bc6-3bd052e05dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:25.843743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86642ea4-22e6-4c44-9bc6-3bd052e05dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:25.894698 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86642ea4-22e6-4c44-9bc6-3bd052e05dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:25.896151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86642ea4-22e6-4c44-9bc6-3bd052e05dbd service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:25.905801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1239/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:33.440705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:33.506067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a7be6-ee21-4412-85e8-0d450f475a0a service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:33.511386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:33.511955 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:33.512264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:33.512539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] PATH_INFO: `/v3/users/03157413068a467cb049c3f9749270e9` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:33.513896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03157413068a467cb049c3f9749270e9)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:33.515957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:33.666777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03157413068a467cb049c3f9749270e9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:33.725881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03157413068a467cb049c3f9749270e9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:33.739532 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03157413068a467cb049c3f9749270e9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:33.740253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03157413068a467cb049c3f9749270e9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:33.740842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03157413068a467cb049c3f9749270e9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:33.758147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1255/2494] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:33 2022] DELETE /identity/v3/users/03157413068a467cb049c3f9749270e9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:33.768509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:33.768982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:33.768982 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:33.788240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:33.789701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:33.854071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1240/2495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:33.865424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:33.866954 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:33.891969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:33.930946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddcf1b-46d4-4cac-832f-7b3adad7eeb6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:33.936229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:33.939002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:33.939002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:33.939002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:33.948330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:33.950104 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:33.961068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1256/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:34.363045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:34.364697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:34.378843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04080-c103-4033-b879-14d5c76e3109 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:34.384318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:34.385010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:34.385286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:34.385554 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] PATH_INFO: `/v3/projects/12ed26f0b8b14e7ea9d710986bf0b022` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:34.386755 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ed26f0b8b14e7ea9d710986bf0b022)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:34.388666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:34.495329 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:34.498177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ed26f0b8b14e7ea9d710986bf0b022', 'request_id': 'req-07e36948-e985-44e0-8b4a-f3b7c12e9efe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.504368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ed26f0b8b14e7ea9d710986bf0b022', 'request_id': 'req-07e36948-e985-44e0-8b4a-f3b7c12e9efe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.569842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ed26f0b8b14e7ea9d710986bf0b022', 'request_id': 'req-07e36948-e985-44e0-8b4a-f3b7c12e9efe'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.581087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] The token cache is being invalidate because project 12ed26f0b8b14e7ea9d710986bf0b022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:34.581377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.582023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.583591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1241/2497] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:34 2022] DELETE /identity/v3/projects/12ed26f0b8b14e7ea9d710986bf0b022 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:34.587537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:34.597813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e24663-80f7-4668-9339-19f1aed7f2f6 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:34.653914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:34.654527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:34.661712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:34.661712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] PATH_INFO: `/v3/users/6afc104338a54e15889d4d73e52780ec` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:34.661712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6afc104338a54e15889d4d73e52780ec)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:34.661712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:34.825136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6afc104338a54e15889d4d73e52780ec'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.914475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6afc104338a54e15889d4d73e52780ec'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.929181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6afc104338a54e15889d4d73e52780ec'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.929884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6afc104338a54e15889d4d73e52780ec'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.931074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c25ab234-c1ef-4a04-9daf-284b0ea9f1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6afc104338a54e15889d4d73e52780ec'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:34.942833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1257/2498] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:34 2022] DELETE /identity/v3/users/6afc104338a54e15889d4d73e52780ec => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:35.362434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:35.366179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:35.377980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e36948-e985-44e0-8b4a-f3b7c12e9efe admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:35.434033 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:35.434623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:35.434897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:35.435159 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] PATH_INFO: `/v3/projects/a958764554444d23803dc51a1ca680f4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:35.436258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a958764554444d23803dc51a1ca680f4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:35.438155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:35.548728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a958764554444d23803dc51a1ca680f4', 'request_id': 'req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:35.554819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a958764554444d23803dc51a1ca680f4', 'request_id': 'req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:35.617192 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a958764554444d23803dc51a1ca680f4', 'request_id': 'req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:35.628401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] The token cache is being invalidate because project a958764554444d23803dc51a1ca680f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:35.631298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:35.631298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fbc463ef-357d-4dd4-800f-8b0083d0b3b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:35.631298 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1242/2499] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:35 2022] DELETE /identity/v3/projects/a958764554444d23803dc51a1ca680f4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:35.636462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c9b46-4cab-4569-b3a4-d7c35f7f958a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:35.636967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c9b46-4cab-4569-b3a4-d7c35f7f958a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:35.637134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c9b46-4cab-4569-b3a4-d7c35f7f958a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:35.656288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-316c9b46-4cab-4569-b3a4-d7c35f7f958a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:35.657486 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-316c9b46-4cab-4569-b3a4-d7c35f7f958a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:35.696168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:35.717139 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1258/2500] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:35.724903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e23be3-cf5d-4b7d-b336-2fd3ab554c06 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:35.725188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e23be3-cf5d-4b7d-b336-2fd3ab554c06 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:35.725464 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e23be3-cf5d-4b7d-b336-2fd3ab554c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:35.745724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-20e23be3-cf5d-4b7d-b336-2fd3ab554c06 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:35.746959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-20e23be3-cf5d-4b7d-b336-2fd3ab554c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:35.754942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1243/2501] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:35.759154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c9b46-4cab-4569-b3a4-d7c35f7f958a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:35.769022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c9b46-4cab-4569-b3a4-d7c35f7f958a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:35.773612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:35.774198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:35.774520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:35.774794 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:35.775435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:41:35.775738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:35.778170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:35.790024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1259/2502] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:41:35.794845 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e23be3-cf5d-4b7d-b336-2fd3ab554c06 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:35.804030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e23be3-cf5d-4b7d-b336-2fd3ab554c06 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:35.808793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:35.809380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:35.809657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:35.811318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:35.811318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:35.812433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:35.961934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1244/2503] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:41:35 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:35.966646 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:35.977030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa2e61-8a11-4ef1-afb9-3fff4ea68017 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.033326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.033933 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.034245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.034554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.035211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.037178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.114760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1260/2504] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:41:35 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:36.119083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.130381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c6cab-a2a3-49c9-8821-501a9f583aac admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.134967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.136671 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.136955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.137223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.137859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.139517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.149543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1245/2505] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:36.154282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.164310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b46af9-a926-46b1-a3bb-d2cfda04c3a9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.168941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597605c-c7df-447a-9942-e0b724e77359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.169529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597605c-c7df-447a-9942-e0b724e77359 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.169807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597605c-c7df-447a-9942-e0b724e77359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.170080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597605c-c7df-447a-9942-e0b724e77359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.170762 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597605c-c7df-447a-9942-e0b724e77359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.172345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597605c-c7df-447a-9942-e0b724e77359 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.184108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1261/2506] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:36.188257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.197969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc2f83-10d0-4291-992f-6d46ae4eab89 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.203804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.204430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.204707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.204976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] PATH_INFO: `/v3/projects/946f67dc6f2c47c9b406832508ebe38e/users/8c2891340f0946a99140a9fc9702cadf/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.220255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946f67dc6f2c47c9b406832508ebe38e, user_id=8c2891340f0946a99140a9fc9702cadf, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.222868 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.275838 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:36.281313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1246/2507] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:36 2022] PUT /identity/v3/projects/946f67dc6f2c47c9b406832508ebe38e/users/8c2891340f0946a99140a9fc9702cadf/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:36.291495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ea917-dd49-4b57-b2ba-c35eb6be0c65 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.291898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ea917-dd49-4b57-b2ba-c35eb6be0c65 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.292456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ea917-dd49-4b57-b2ba-c35eb6be0c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.319128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8f9ea917-dd49-4b57-b2ba-c35eb6be0c65 None None] MFA Rules not processed for user `8c2891340f0946a99140a9fc9702cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:36.320369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8f9ea917-dd49-4b57-b2ba-c35eb6be0c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:36.384584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1262/2508] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:36.389555 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.450515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a571-f97b-451c-b774-8e72c3a86207 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.455528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.456115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.456388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.456657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.457266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.459209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.542506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1247/2509] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:41:36 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:36.548891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ea917-dd49-4b57-b2ba-c35eb6be0c65 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.557194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ea917-dd49-4b57-b2ba-c35eb6be0c65 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.591711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:36.614613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.615209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.615485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.615750 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.616382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.618270 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.719307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1263/2510] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:41:36 2022] POST /identity/v3/users => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:36.723199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.733508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d592f16-c5c7-4a2d-a611-2399f73a9c56 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.738150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.738777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.739050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.739318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.739965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.741528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.752551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1248/2511] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:36.756586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.767729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60605678-3fa2-47e6-b7ef-1db168b31764 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.772642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.773285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.773562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.776090 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] PATH_INFO: `/v3/projects/81c305ce5ad147b5845c0b0ddc84a66c/users/5ba4befb743642349ab50d7d22ea297c/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.788835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c305ce5ad147b5845c0b0ddc84a66c, user_id=5ba4befb743642349ab50d7d22ea297c, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.791492 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.856537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1264/2512] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:36 2022] PUT /identity/v3/projects/81c305ce5ad147b5845c0b0ddc84a66c/users/5ba4befb743642349ab50d7d22ea297c/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:36.860431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.871593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a1f6-9149-49cf-a5ce-c9664944128e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.927218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84271bc-7ebd-4522-aac4-58ac2dd3d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.927820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84271bc-7ebd-4522-aac4-58ac2dd3d8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.928094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84271bc-7ebd-4522-aac4-58ac2dd3d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.928362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84271bc-7ebd-4522-aac4-58ac2dd3d8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.929000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84271bc-7ebd-4522-aac4-58ac2dd3d8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.930627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84271bc-7ebd-4522-aac4-58ac2dd3d8b9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.941844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1249/2513] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:36.945951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:36.956645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3248a76-f4c1-45de-99e1-a04b1a3577b5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:36.961216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:36.961859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.962136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.962443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] PATH_INFO: `/v3/projects/81c305ce5ad147b5845c0b0ddc84a66c/users/5ba4befb743642349ab50d7d22ea297c/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:36.964376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c305ce5ad147b5845c0b0ddc84a66c, user_id=5ba4befb743642349ab50d7d22ea297c, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:36.967028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:36.977914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1265/2514] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:36 2022] PUT /identity/v3/projects/81c305ce5ad147b5845c0b0ddc84a66c/users/5ba4befb743642349ab50d7d22ea297c/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:36.990592 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:36.991045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:36.991343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:37.025451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] MFA Rules not processed for user `5ba4befb743642349ab50d7d22ea297c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:37.027065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:37.092091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:37.100821 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1250/2515] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:37.112250 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:37.113908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:37.171685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792dfa-458f-4c2d-ba3d-e0dd4abd2aa4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:37.175344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:37.175932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:37.176210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:37.176481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:37.232202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:37.233617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:37.242312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1266/2516] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:42.313950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:42.315447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:42.327989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5d941-4a15-42b5-9e3e-c18b7e4e7066 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:42.331537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45df8b-d9af-4e05-82db-ed79fb36f230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:42.332127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45df8b-d9af-4e05-82db-ed79fb36f230 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:42.332403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45df8b-d9af-4e05-82db-ed79fb36f230 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:42.332670 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45df8b-d9af-4e05-82db-ed79fb36f230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:42.341483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45df8b-d9af-4e05-82db-ed79fb36f230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:42.343614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45df8b-d9af-4e05-82db-ed79fb36f230 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:42.353222 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1251/2517] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:47.238345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:48.222874 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:48.284835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16184a-9c30-4fa4-84f7-6bfb2c7911bf service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:48.289433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:48.290045 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:48.290323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:48.290622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] PATH_INFO: `/v3/users/8c2891340f0946a99140a9fc9702cadf` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:48.291732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c2891340f0946a99140a9fc9702cadf)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:48.293633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:48.441617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c2891340f0946a99140a9fc9702cadf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:48.485705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c2891340f0946a99140a9fc9702cadf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:48.497724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c2891340f0946a99140a9fc9702cadf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:48.498367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c2891340f0946a99140a9fc9702cadf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:48.499125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c2891340f0946a99140a9fc9702cadf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:48.511274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1267/2518] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:48 2022] DELETE /identity/v3/users/8c2891340f0946a99140a9fc9702cadf => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:48.515794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:48.516285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:48.516395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:48.535654 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:48.536858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:48.594557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1252/2519] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:48.604124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:48.605513 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:48.661944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2db048-9be4-40ce-b3e1-f3206058f497 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:48.665601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:48.666187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:48.666500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:48.666775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:48.676630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:48.678049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:48.687560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1268/2520] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:49.065584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:49.067258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:49.082229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d79c7-7f2b-4d1b-9634-de573464b1d6 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:49.088175 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:49.088922 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:49.089211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:49.089523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] PATH_INFO: `/v3/projects/946f67dc6f2c47c9b406832508ebe38e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:49.090702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] RBAC: Authorizing `identity:delete_project(project_id=946f67dc6f2c47c9b406832508ebe38e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:49.093480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:49.193597 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946f67dc6f2c47c9b406832508ebe38e', 'request_id': 'req-65843185-c50f-4ae0-ab1f-4b66ff56f86e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.199678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946f67dc6f2c47c9b406832508ebe38e', 'request_id': 'req-65843185-c50f-4ae0-ab1f-4b66ff56f86e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.290023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946f67dc6f2c47c9b406832508ebe38e', 'request_id': 'req-65843185-c50f-4ae0-ab1f-4b66ff56f86e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.300583 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] The token cache is being invalidate because project 946f67dc6f2c47c9b406832508ebe38e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:49.300870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.301571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.303077 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1253/2521] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:49 2022] DELETE /identity/v3/projects/946f67dc6f2c47c9b406832508ebe38e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:49.307169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:49.317371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba954d7-8319-4bf9-89f0-e9c807440982 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:49.335609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:49.374960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:49.375599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:49.375906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:49.376178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] PATH_INFO: `/v3/users/5ba4befb743642349ab50d7d22ea297c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:49.377309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba4befb743642349ab50d7d22ea297c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:49.379299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:49.566795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba4befb743642349ab50d7d22ea297c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.626895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba4befb743642349ab50d7d22ea297c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.638905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba4befb743642349ab50d7d22ea297c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.640148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba4befb743642349ab50d7d22ea297c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.640771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55236407-8005-4cce-9302-2fa0a333b7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba4befb743642349ab50d7d22ea297c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:49.652624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1269/2522] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:49 2022] DELETE /identity/v3/users/5ba4befb743642349ab50d7d22ea297c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:50.069919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:50.071345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:50.095987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65843185-c50f-4ae0-ab1f-4b66ff56f86e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:50.154959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:50.155906 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.156233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.156513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] PATH_INFO: `/v3/projects/81c305ce5ad147b5845c0b0ddc84a66c` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.157663 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=81c305ce5ad147b5845c0b0ddc84a66c)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:50.159603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:50.167876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:50.256833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81c305ce5ad147b5845c0b0ddc84a66c', 'request_id': 'req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:50.262888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81c305ce5ad147b5845c0b0ddc84a66c', 'request_id': 'req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:50.341253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81c305ce5ad147b5845c0b0ddc84a66c', 'request_id': 'req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:50.354305 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] The token cache is being invalidate because project 81c305ce5ad147b5845c0b0ddc84a66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:50.354634 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:50.355295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-97f5cda7-1179-4162-ac40-0ccc2a8f50bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:50.356935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1254/2523] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:50 2022] DELETE /identity/v3/projects/81c305ce5ad147b5845c0b0ddc84a66c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:50.363556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c5e9-b4c8-4c10-a18b-3ecc6a431f85 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.364040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c5e9-b4c8-4c10-a18b-3ecc6a431f85 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.364184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99c5e9-b4c8-4c10-a18b-3ecc6a431f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.384881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0f99c5e9-b4c8-4c10-a18b-3ecc6a431f85 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:50.386151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0f99c5e9-b4c8-4c10-a18b-3ecc6a431f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:50.445897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1270/2524] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:50.454948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7788c-4369-43bd-9215-afb606cc5230 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.455235 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7788c-4369-43bd-9215-afb606cc5230 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.455508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7788c-4369-43bd-9215-afb606cc5230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.475045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0ed7788c-4369-43bd-9215-afb606cc5230 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:50.476233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0ed7788c-4369-43bd-9215-afb606cc5230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:50.484790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1255/2525] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:50.496217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99c5e9-b4c8-4c10-a18b-3ecc6a431f85 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:50.506756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99c5e9-b4c8-4c10-a18b-3ecc6a431f85 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:50.511351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:50.511937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.512216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.512485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.513151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:41:50.513452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:50.515217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:50.526537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1271/2526] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:41:50.531373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7788c-4369-43bd-9215-afb606cc5230 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:50.540730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7788c-4369-43bd-9215-afb606cc5230 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:50.545521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:50.551941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.551941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.551941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.551941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:50.551941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:50.651984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1256/2527] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:41:50 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:50.658763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:50.669487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852beed-9a7f-4af4-830a-60defda248bc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:50.697025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:50.727666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:50.728252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.728526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.728796 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.729415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:50.731336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:50.805904 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1272/2528] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:41:50 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:41:50.810141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:50.820416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533067b-98ed-416b-8e85-4694915da818 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:50.825036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:50.829559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.829559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.829559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.829559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:50.829559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:50.839373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1257/2529] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:41:50.847472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:50.853524 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f5399-0e7a-48f2-8be0-efe179896bcb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:50.858169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f7a9a-d7ac-40d2-963b-2ca76d125e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:50.859621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7a9a-d7ac-40d2-963b-2ca76d125e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.859898 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7a9a-d7ac-40d2-963b-2ca76d125e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.860164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7a9a-d7ac-40d2-963b-2ca76d125e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.860792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f7a9a-d7ac-40d2-963b-2ca76d125e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:50.862594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f7a9a-d7ac-40d2-963b-2ca76d125e16 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:50.872629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1273/2530] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:41:50.877074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:50.887340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89bce7-84fe-4cf6-a173-b129876f936d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:50.892168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:50.892810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:50.893088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:50.893359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] PATH_INFO: `/v3/projects/5bbb3b8f0796413aa29745ac985d2f89/users/f18c6db4cdbc4842aa9ca3549ccdf9ea/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:50.909518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbb3b8f0796413aa29745ac985d2f89, user_id=f18c6db4cdbc4842aa9ca3549ccdf9ea, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:50.912252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:50.993295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1258/2531] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:41:50 2022] PUT /identity/v3/projects/5bbb3b8f0796413aa29745ac985d2f89/users/f18c6db4cdbc4842aa9ca3549ccdf9ea/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:41:51.004462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:51.004741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:51.005034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:51.034443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] MFA Rules not processed for user `f18c6db4cdbc4842aa9ca3549ccdf9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:41:51.035642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:51.099943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1274/2532] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:41:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:41:51.111815 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:51.113581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:51.169881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc198b0-03bd-4fd5-8c53-5c65f4955480 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:51.173881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bddf4-20d4-436b-8a4f-32ad01fac66e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:51.174576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bddf4-20d4-436b-8a4f-32ad01fac66e service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:51.174857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bddf4-20d4-436b-8a4f-32ad01fac66e service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:51.175125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bddf4-20d4-436b-8a4f-32ad01fac66e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:51.183331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bddf4-20d4-436b-8a4f-32ad01fac66e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:51.184750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bddf4-20d4-436b-8a4f-32ad01fac66e service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:51.193359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1259/2533] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:01.096926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:01.189546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:13.193118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:13.194746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:13.253335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695b4d5-ce5e-478d-9511-368dfb647be6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:13.257307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:13.257905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:13.258179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:13.258475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:13.260220 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:13.335142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:13.336500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:13.357076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1275/2534] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:16.141611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:16.142102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:16.142333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:16.176587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:16.178732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:16.235449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1260/2535] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:42:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:16.357922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:16.359462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:16.431096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f0cd1-c69d-423f-9cda-1ecdccc9f4d9 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:16.436228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c46dd7-eeba-4e96-b8f0-855d9d085279 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:16.436820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c46dd7-eeba-4e96-b8f0-855d9d085279 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:16.437103 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c46dd7-eeba-4e96-b8f0-855d9d085279 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:16.437375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c46dd7-eeba-4e96-b8f0-855d9d085279 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:16.445585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c46dd7-eeba-4e96-b8f0-855d9d085279 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:16.447208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c46dd7-eeba-4e96-b8f0-855d9d085279 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:16.455580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1276/2536] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:42:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:42:16.549440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:16.550894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:16.569813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c562f-3a0d-436a-bcf6-fd60f53f3b91 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:16.574682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:16.575524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:16.576403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:16.577081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:16.579025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:16.607610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:16.609084 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:16.626195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1261/2537] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:19.373744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c8ed-8778-4666-9ffa-9560985ba60b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:19.374399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c8ed-8778-4666-9ffa-9560985ba60b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:19.374399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c8ed-8778-4666-9ffa-9560985ba60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:19.395484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e7b4c8ed-8778-4666-9ffa-9560985ba60b None None] MFA Rules not processed for user `55afeedd651248da8ac80c61fb0729cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:19.397358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e7b4c8ed-8778-4666-9ffa-9560985ba60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:19.425277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:19.456901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1277/2538] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:20.046482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:20.122974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba7d9-7159-4dd5-b601-85a487728fa0 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:20.127978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:20.128731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:20.129047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:20.129373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] PATH_INFO: `/v3/users/f18c6db4cdbc4842aa9ca3549ccdf9ea` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:20.131402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f18c6db4cdbc4842aa9ca3549ccdf9ea)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:20.133888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:20.630792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f18c6db4cdbc4842aa9ca3549ccdf9ea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:20.697650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f18c6db4cdbc4842aa9ca3549ccdf9ea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:20.717254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f18c6db4cdbc4842aa9ca3549ccdf9ea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:20.731859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f18c6db4cdbc4842aa9ca3549ccdf9ea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:20.732585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f18c6db4cdbc4842aa9ca3549ccdf9ea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:20.740772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1262/2539] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:42:20 2022] DELETE /identity/v3/users/f18c6db4cdbc4842aa9ca3549ccdf9ea => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:20.744899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:20.745356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:20.745440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:20.767656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:20.769269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:20.853080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1278/2540] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:42:20.864149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:20.866007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:20.872522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:20.950455 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd575b-f3f7-44bf-ab30-c38483e1ccf7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:20.954701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43465131-70f7-4047-8d2e-aa304c21f1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:20.955324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43465131-70f7-4047-8d2e-aa304c21f1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:20.955619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43465131-70f7-4047-8d2e-aa304c21f1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:20.955946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43465131-70f7-4047-8d2e-aa304c21f1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:20.970908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43465131-70f7-4047-8d2e-aa304c21f1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:20.972368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43465131-70f7-4047-8d2e-aa304c21f1f1 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:20.984954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1263/2541] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:21.419305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:21.422789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:21.454805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22b998-2bb9-4f54-90b7-5dcaf5365d50 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:21.469422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:21.470144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:21.470454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:21.470728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] PATH_INFO: `/v3/projects/5bbb3b8f0796413aa29745ac985d2f89` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:21.474176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bbb3b8f0796413aa29745ac985d2f89)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:21.479634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:21.612806 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bbb3b8f0796413aa29745ac985d2f89', 'request_id': 'req-99c7fecb-3515-402f-9295-6572afdc69f3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:21.670151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bbb3b8f0796413aa29745ac985d2f89', 'request_id': 'req-99c7fecb-3515-402f-9295-6572afdc69f3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:21.688066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bbb3b8f0796413aa29745ac985d2f89', 'request_id': 'req-99c7fecb-3515-402f-9295-6572afdc69f3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:21.710185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] The token cache is being invalidate because project 5bbb3b8f0796413aa29745ac985d2f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:42:21.710667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:21.711268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-99c7fecb-3515-402f-9295-6572afdc69f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:21.712919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1279/2542] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:42:21 2022] DELETE /identity/v3/projects/5bbb3b8f0796413aa29745ac985d2f89 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:21.717886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d47bf-6a25-460c-901d-f38fcb23177e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:21.718398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d47bf-6a25-460c-901d-f38fcb23177e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:21.718501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d47bf-6a25-460c-901d-f38fcb23177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:21.755335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-dc0d47bf-6a25-460c-901d-f38fcb23177e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:21.756833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dc0d47bf-6a25-460c-901d-f38fcb23177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:21.839956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1264/2543] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:21.847753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c772b0a-9cb9-4c6e-9534-636aa3d3d7fb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:21.848301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c772b0a-9cb9-4c6e-9534-636aa3d3d7fb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:21.849014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c772b0a-9cb9-4c6e-9534-636aa3d3d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:21.870784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7c772b0a-9cb9-4c6e-9534-636aa3d3d7fb None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:21.872156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7c772b0a-9cb9-4c6e-9534-636aa3d3d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:21.881286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1280/2544] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:42:21.885393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d47bf-6a25-460c-901d-f38fcb23177e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:21.899337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d47bf-6a25-460c-901d-f38fcb23177e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:21.904499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:21.905131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:21.905418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:21.905693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:21.906684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:42:21.907112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:21.908717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:21.928987 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1265/2545] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:42:21.933158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c772b0a-9cb9-4c6e-9534-636aa3d3d7fb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:21.944850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c772b0a-9cb9-4c6e-9534-636aa3d3d7fb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:21.950063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:21.950717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:21.951288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:21.951425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:21.952225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:21.954160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:22.110201 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1281/2546] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:42:21 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:42:22.111157 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:22.123892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f487321-b9f8-4483-b040-3e0421d65ffb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:22.202813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:22.203340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:22.203616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:22.203880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:22.204503 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:22.206442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:22.288053 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1266/2547] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:42:22 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:42:22.292063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:22.304931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dde9e1-5e90-461e-8a0f-a69a4110419f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:22.311259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:22.312066 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:22.312475 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:22.313316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:22.313568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:22.318826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:22.329764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1282/2548] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:22.331013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:22.347249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aca421-fa85-453e-93a5-9a57da7d73b5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:22.354397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062957d6-e79e-464b-80b5-df17a78b90ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:22.355034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062957d6-e79e-464b-80b5-df17a78b90ff admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:22.355310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062957d6-e79e-464b-80b5-df17a78b90ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:22.355580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062957d6-e79e-464b-80b5-df17a78b90ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:22.356215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062957d6-e79e-464b-80b5-df17a78b90ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:22.357832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062957d6-e79e-464b-80b5-df17a78b90ff admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:22.373354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1267/2549] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:22.377834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:22.388364 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71d4ff-415d-4870-bd47-323382948d86 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:22.394871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:22.396092 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:22.396372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:22.396769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] PATH_INFO: `/v3/projects/f960d55de962470990763bb7cd4621c7/users/20c488f6e03a43bc80a64e403600db1f/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:22.406048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:22.415029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f960d55de962470990763bb7cd4621c7, user_id=20c488f6e03a43bc80a64e403600db1f, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:22.418529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:22.495028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1283/2550] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:42:22 2022] PUT /identity/v3/projects/f960d55de962470990763bb7cd4621c7/users/20c488f6e03a43bc80a64e403600db1f/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:42:22.497054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:22.497511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:22.497912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:22.519408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:22.528354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] MFA Rules not processed for user `20c488f6e03a43bc80a64e403600db1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:22.529596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:22.599677 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1268/2551] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:22.612519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:22.615003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:22.687666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:22.689170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:22.702756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479e1f-7512-4041-812c-f83df0867cbc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:22.707337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0ebbf-5a60-4a11-a27d-7c3c34dc8f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:22.709363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ebbf-5a60-4a11-a27d-7c3c34dc8f5e service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:22.710212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ebbf-5a60-4a11-a27d-7c3c34dc8f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:22.710673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ebbf-5a60-4a11-a27d-7c3c34dc8f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:22.720828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0ebbf-5a60-4a11-a27d-7c3c34dc8f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:22.722545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0ebbf-5a60-4a11-a27d-7c3c34dc8f5e service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:22.738422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1284/2552] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:22.870480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bb844-caad-4cac-8a87-e9270fb9cdf8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:22.878612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:22.878769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:22.878942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:22.879368 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:22.885180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:23.071341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:23.073088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:23.110539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1269/2553] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:32.729822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:33.101152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:49.275095 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8d0f0-ce18-4a52-816c-63030ac85834 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:49.275579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8d0f0-ce18-4a52-816c-63030ac85834 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:49.275673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8d0f0-ce18-4a52-816c-63030ac85834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:49.331500 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-00e8d0f0-ce18-4a52-816c-63030ac85834 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:49.332519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-00e8d0f0-ce18-4a52-816c-63030ac85834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:49.434773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1285/2554] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:42:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:49.434991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:49.434991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:49.513022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed0ebc-c4e4-4519-9ba6-c20f8e01ed53 service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:49.518435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196426e3-1cea-4f6e-8f27-ebe44fa52fc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:49.519127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196426e3-1cea-4f6e-8f27-ebe44fa52fc9 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:49.519402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196426e3-1cea-4f6e-8f27-ebe44fa52fc9 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:49.519668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196426e3-1cea-4f6e-8f27-ebe44fa52fc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:49.529502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196426e3-1cea-4f6e-8f27-ebe44fa52fc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:49.531217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196426e3-1cea-4f6e-8f27-ebe44fa52fc9 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:49.540812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1270/2555] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:42:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:42:51.426738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8d0f0-ce18-4a52-816c-63030ac85834 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:51.499088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8d0f0-ce18-4a52-816c-63030ac85834 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:51.503808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:51.504438 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:51.504711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:51.504979 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] PATH_INFO: `/v3/users/20c488f6e03a43bc80a64e403600db1f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:51.506324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c488f6e03a43bc80a64e403600db1f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:51.508526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:51.745435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c488f6e03a43bc80a64e403600db1f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:51.828072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c488f6e03a43bc80a64e403600db1f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:51.842108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c488f6e03a43bc80a64e403600db1f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:51.842862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c488f6e03a43bc80a64e403600db1f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:51.843769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c488f6e03a43bc80a64e403600db1f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:51.846478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:51.859749 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1286/2556] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:42:51 2022] DELETE /identity/v3/users/20c488f6e03a43bc80a64e403600db1f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:51.865602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:51.865602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:51.865602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:51.892630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:51.894094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:51.973414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1271/2557] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:42:51.987234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:51.989046 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:52.108925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2937eb-b67e-4fa0-a504-290be01c494f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:52.119479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327a5fd-b50c-485d-8c3c-6afcd0bd517f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:52.121895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a5fd-b50c-485d-8c3c-6afcd0bd517f service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:52.122396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a5fd-b50c-485d-8c3c-6afcd0bd517f service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:52.122871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327a5fd-b50c-485d-8c3c-6afcd0bd517f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:52.180671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327a5fd-b50c-485d-8c3c-6afcd0bd517f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:52.182234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327a5fd-b50c-485d-8c3c-6afcd0bd517f service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:52.193873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1287/2558] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:52.594111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:52.597745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:52.632995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952df982-0bd5-417b-b9fa-b91e3b34e197 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:52.638969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:52.639603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:52.640473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:52.640754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] PATH_INFO: `/v3/projects/f960d55de962470990763bb7cd4621c7` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:52.642287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f960d55de962470990763bb7cd4621c7)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:52.644374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:52.757314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f960d55de962470990763bb7cd4621c7', 'request_id': 'req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:52.765642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f960d55de962470990763bb7cd4621c7', 'request_id': 'req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:52.844829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f960d55de962470990763bb7cd4621c7', 'request_id': 'req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:52.850025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:52.858731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] The token cache is being invalidate because project f960d55de962470990763bb7cd4621c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:42:52.859032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:52.860699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f520c4ae-28fe-4af5-8b7c-6b9acd0c8bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:42:52.862213 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1272/2559] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:42:52 2022] DELETE /identity/v3/projects/f960d55de962470990763bb7cd4621c7 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:52.867179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85cbe0-b177-4988-b859-d5d1fb6bb82e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:52.867666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85cbe0-b177-4988-b859-d5d1fb6bb82e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:52.867784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85cbe0-b177-4988-b859-d5d1fb6bb82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:52.888305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cc85cbe0-b177-4988-b859-d5d1fb6bb82e None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:52.889578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cc85cbe0-b177-4988-b859-d5d1fb6bb82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:52.963015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1288/2560] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:53.029494 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdf39-2953-49e5-8eb3-10939699446d None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.030056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdf39-2953-49e5-8eb3-10939699446d None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.030056 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdf39-2953-49e5-8eb3-10939699446d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.051360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-da1cdf39-2953-49e5-8eb3-10939699446d None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:53.053232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-da1cdf39-2953-49e5-8eb3-10939699446d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:53.061518 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1273/2561] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:53.065889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85cbe0-b177-4988-b859-d5d1fb6bb82e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:53.081509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85cbe0-b177-4988-b859-d5d1fb6bb82e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:53.086697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:53.087313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.087595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.088034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.089930 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:42:53.090272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:53.092040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:53.107672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1289/2562] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:42:53.110636 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdf39-2953-49e5-8eb3-10939699446d None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:53.121105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdf39-2953-49e5-8eb3-10939699446d None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:53.125925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:53.126772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.126898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.127285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.127929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:53.130129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:53.215745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1274/2563] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:42:53 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:42:53.220274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:53.233240 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905c28c-56aa-407e-a99d-1b4569cbffbe admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:53.353708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:53.354338 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.354648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.354922 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.355553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:53.357851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:53.531215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1290/2564] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:42:53 2022] POST /identity/v3/users => generated 292 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:42:53.535587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:53.547871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12244e10-8935-4327-83b8-1032ebaaec0f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:53.553321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:53.554102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.554480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.554810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.555553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:53.557170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:53.570377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1275/2565] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:53.571987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:53.582887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb1510-e7f9-4c2a-a411-89f8ef9a7cd7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:53.588671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19d456-acaf-4ef1-b886-24ec5a7d48fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:53.589355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19d456-acaf-4ef1-b886-24ec5a7d48fc admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.589641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19d456-acaf-4ef1-b886-24ec5a7d48fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.589989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19d456-acaf-4ef1-b886-24ec5a7d48fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.590895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19d456-acaf-4ef1-b886-24ec5a7d48fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:53.592609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19d456-acaf-4ef1-b886-24ec5a7d48fc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:53.599161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:53.606966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1291/2566] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:53.618300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:53.630335 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb5e6b-8631-4431-92ed-a28df00fa0dc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:53.635507 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:53.636165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.636440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.641537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] PATH_INFO: `/v3/projects/9ec96b76ebd04f818d881dd3641a240c/users/de7e2695f1c3443e9cf1a0adbdc66650/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.655231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec96b76ebd04f818d881dd3641a240c, user_id=de7e2695f1c3443e9cf1a0adbdc66650, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:53.664150 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:53.719514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1276/2567] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:42:53 2022] PUT /identity/v3/projects/9ec96b76ebd04f818d881dd3641a240c/users/de7e2695f1c3443e9cf1a0adbdc66650/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:53.730882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1159d6-8c2b-444c-861d-1f07dd1410c8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.731235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1159d6-8c2b-444c-861d-1f07dd1410c8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.731508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1159d6-8c2b-444c-861d-1f07dd1410c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.770996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9b1159d6-8c2b-444c-861d-1f07dd1410c8 None None] MFA Rules not processed for user `de7e2695f1c3443e9cf1a0adbdc66650`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:53.772797 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b1159d6-8c2b-444c-861d-1f07dd1410c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:53.846554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1292/2568] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:53.852793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:53.939313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a593-1564-40d8-8534-57ab4ed27c4a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:53.945319 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:53.945934 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:53.946489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:53.946767 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:53.947399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:53.949326 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.047983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1277/2569] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:42:53 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:42:54.052118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1159d6-8c2b-444c-861d-1f07dd1410c8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.064783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1159d6-8c2b-444c-861d-1f07dd1410c8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.125699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.126309 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.126620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.126893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.127561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.129448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.217163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1293/2570] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:42:54 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:42:54.221182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.233205 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16d8fa-2620-4bac-b4a4-8698ef6122a9 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.238074 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.238715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.238991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.239268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.240040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.242447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.255491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1278/2571] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:54.260462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.270726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4758e-ef89-4d71-a052-841490b0bb86 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.276366 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef89e8-90a4-403c-83e8-cc24823c220c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.276965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef89e8-90a4-403c-83e8-cc24823c220c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.277247 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef89e8-90a4-403c-83e8-cc24823c220c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.277546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef89e8-90a4-403c-83e8-cc24823c220c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.278165 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef89e8-90a4-403c-83e8-cc24823c220c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.279805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef89e8-90a4-403c-83e8-cc24823c220c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.294927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1294/2572] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:54.296373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.308020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6231f9c-c2b6-42bd-a3bd-f9c47698e388 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.312689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.313339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.313620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.313889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] PATH_INFO: `/v3/projects/5fc28d14e7cb46a983443a2ca47d862e/users/e79f276c45c84dbb8243a69f06000b14/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.330499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc28d14e7cb46a983443a2ca47d862e, user_id=e79f276c45c84dbb8243a69f06000b14, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.337120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.412920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1279/2573] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:42:54 2022] PUT /identity/v3/projects/5fc28d14e7cb46a983443a2ca47d862e/users/e79f276c45c84dbb8243a69f06000b14/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:54.424951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8ac1-9b83-4f12-9a31-bd2a25893969 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.425322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8ac1-9b83-4f12-9a31-bd2a25893969 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.425672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8ac1-9b83-4f12-9a31-bd2a25893969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.485144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1f5f8ac1-9b83-4f12-9a31-bd2a25893969 None None] MFA Rules not processed for user `e79f276c45c84dbb8243a69f06000b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:54.487451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1f5f8ac1-9b83-4f12-9a31-bd2a25893969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:54.566608 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:54.572518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1295/2574] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:54.577411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.633881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:54.662608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771dfa-46a5-45fd-92b9-bd5b85fdc5e0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.669132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.669744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.670022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.670291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.670968 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.673026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.797580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1280/2575] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:42:54 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:42:54.803123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f8ac1-9b83-4f12-9a31-bd2a25893969 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.811496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f8ac1-9b83-4f12-9a31-bd2a25893969 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.876642 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.877792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.878296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.878997 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.879996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.883669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:54.971496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1296/2576] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:42:54 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:42:54.976282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:54.987369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3de18-fa55-449f-84ef-4f6c8d3b66cc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:54.993087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:54.994058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:54.994332 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:54.994625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:54.995267 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:54.996951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.007835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1281/2577] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:42:55.012336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.022614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0d3dc-8f25-496f-b75a-6245afc39bdd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.027180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.027819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.028093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.028361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] PATH_INFO: `/v3/projects/f8cfcc1b071f4a57a34cadef1135c8d7/users/6719d09e7e9243cb90029d01d0c5aa92/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.044781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cfcc1b071f4a57a34cadef1135c8d7, user_id=6719d09e7e9243cb90029d01d0c5aa92, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.047881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.103790 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1297/2578] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:42:55 2022] PUT /identity/v3/projects/f8cfcc1b071f4a57a34cadef1135c8d7/users/6719d09e7e9243cb90029d01d0c5aa92/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:42:55.108113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.120666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbe768-f725-4815-b5a1-821662468900 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.181736 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293d495-d171-4fb9-a316-99fb0c4aa510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.182401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293d495-d171-4fb9-a316-99fb0c4aa510 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.182684 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293d495-d171-4fb9-a316-99fb0c4aa510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.183017 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293d495-d171-4fb9-a316-99fb0c4aa510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.183660 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293d495-d171-4fb9-a316-99fb0c4aa510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.185313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293d495-d171-4fb9-a316-99fb0c4aa510 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.199238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1282/2579] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:42:55.200640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.212124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fd736-2cbb-4408-8a64-0805a33a5987 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.217678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.218344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.218663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.218943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] PATH_INFO: `/v3/projects/f8cfcc1b071f4a57a34cadef1135c8d7/users/6719d09e7e9243cb90029d01d0c5aa92/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.220938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cfcc1b071f4a57a34cadef1135c8d7, user_id=6719d09e7e9243cb90029d01d0c5aa92, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.223643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.235641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1298/2580] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:42:55 2022] PUT /identity/v3/projects/f8cfcc1b071f4a57a34cadef1135c8d7/users/6719d09e7e9243cb90029d01d0c5aa92/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:42:55.247046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.247272 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.247551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.283866 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] MFA Rules not processed for user `6719d09e7e9243cb90029d01d0c5aa92`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:42:55.285252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:55.360897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1283/2581] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:42:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:42:55.376420 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:55.378495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:55.444280 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496e86d-7cf1-4958-b1f5-cee299be647c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:55.448179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:55.448778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:55.449054 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:55.449322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:55.516789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:55.518631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:55.534018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1299/2582] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:57.661016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:57.662620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:42:57.674830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bb32e-008c-4cd6-964e-6353c339a10e None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:57.679034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:57.679641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:57.679918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:57.680185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:57.714966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:57.740334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:57.742682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:57.752923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1284/2583] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:42:58.979586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:42:58.984908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:42:58.988585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74031da7-7ef2-483b-af30-8d72b6a63e24 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:42:58.992900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:42:58.996571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:42:58.996571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:42:58.996571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:42:59.003544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:42:59.004957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:42:59.015322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1300/2584] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:42:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:04.516497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.581946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ac19-118f-43e0-bc38-b369bdf62e69 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:04.586790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:04.587420 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.587697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.587967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] PATH_INFO: `/v3/users/de7e2695f1c3443e9cf1a0adbdc66650` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.589098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7e2695f1c3443e9cf1a0adbdc66650)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:04.591072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:04.763943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7e2695f1c3443e9cf1a0adbdc66650'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.861729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7e2695f1c3443e9cf1a0adbdc66650'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.871055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.873233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.876947 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7e2695f1c3443e9cf1a0adbdc66650'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.885258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7e2695f1c3443e9cf1a0adbdc66650'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.885996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3150ef9c-6b51-4251-b3bc-9fefd5600f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7e2695f1c3443e9cf1a0adbdc66650'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:04.894215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1285/2585] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:04 2022] DELETE /identity/v3/users/de7e2695f1c3443e9cf1a0adbdc66650 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:04.898996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.899524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.899800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.923615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:04.924814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.939614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b4b1b-0a45-40b9-ac35-64ff2f6d5049 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:04.980253 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:04.987324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1286/2586] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:43:04.990620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:04.990620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:04.990620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:04.990919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:04.992700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:04.997703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:04.999202 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:05.005188 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.006633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.011308 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cf240-f9ab-4839-b99f-d561724f2b31 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.015032 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.015705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.015977 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.016239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.017151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1301/2587] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:43:05.027287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.029216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.039860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1287/2588] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:05.640294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.642885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:05.659966 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e04fd-fd7b-4bd1-97fc-386b77b68dc2 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.665632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.671752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.671752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.671752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] PATH_INFO: `/v3/projects/9ec96b76ebd04f818d881dd3641a240c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.671752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec96b76ebd04f818d881dd3641a240c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.671752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:05.790471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec96b76ebd04f818d881dd3641a240c', 'request_id': 'req-bf509dda-bf7c-4e55-9c46-804b2ec79d86'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:05.862214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec96b76ebd04f818d881dd3641a240c', 'request_id': 'req-bf509dda-bf7c-4e55-9c46-804b2ec79d86'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:05.870006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec96b76ebd04f818d881dd3641a240c', 'request_id': 'req-bf509dda-bf7c-4e55-9c46-804b2ec79d86'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:05.882711 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] The token cache is being invalidate because project 9ec96b76ebd04f818d881dd3641a240c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:05.883535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:05.884140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:05.888714 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1302/2589] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:05 2022] DELETE /identity/v3/projects/9ec96b76ebd04f818d881dd3641a240c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:05.890953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:05.902819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8294d28-6962-4964-9988-86f5cb074d77 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:05.979858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:05.980484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:05.980757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:05.981025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] PATH_INFO: `/v3/users/e79f276c45c84dbb8243a69f06000b14` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:05.982149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e79f276c45c84dbb8243a69f06000b14)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:05.984105 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:06.164171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e79f276c45c84dbb8243a69f06000b14'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:06.233723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e79f276c45c84dbb8243a69f06000b14'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:06.239290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:06.240784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:06.249102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e79f276c45c84dbb8243a69f06000b14'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:06.261627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e79f276c45c84dbb8243a69f06000b14'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:06.262942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e79f276c45c84dbb8243a69f06000b14'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:06.277392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1288/2590] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:05 2022] DELETE /identity/v3/users/e79f276c45c84dbb8243a69f06000b14 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:06.290393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:06.315935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf509dda-bf7c-4e55-9c46-804b2ec79d86 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:06.368929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:06.369879 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:06.369976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:06.370234 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:06.372274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:06.387938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:06.390318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:06.398212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1303/2591] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:43:06.996585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:06.998703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:07.085434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989c19a-e712-4089-bf6b-0a26e84d01da admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:07.092942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:07.092942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:07.093266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:07.093364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] PATH_INFO: `/v3/projects/5fc28d14e7cb46a983443a2ca47d862e` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:07.094788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc28d14e7cb46a983443a2ca47d862e)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:07.098221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:07.100433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:07.230020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc28d14e7cb46a983443a2ca47d862e', 'request_id': 'req-30158d40-9dd1-4828-88f7-5369c24e9378'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.237599 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc28d14e7cb46a983443a2ca47d862e', 'request_id': 'req-30158d40-9dd1-4828-88f7-5369c24e9378'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.330515 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc28d14e7cb46a983443a2ca47d862e', 'request_id': 'req-30158d40-9dd1-4828-88f7-5369c24e9378'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.341842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] The token cache is being invalidate because project 5fc28d14e7cb46a983443a2ca47d862e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:07.342136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.342825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-30158d40-9dd1-4828-88f7-5369c24e9378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.344609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1289/2592] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:06 2022] DELETE /identity/v3/projects/5fc28d14e7cb46a983443a2ca47d862e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:07.348456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:07.361237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56443c2-20d7-4343-8510-4a2c8afd6cc0 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:07.426916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:07.427647 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:07.427992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:07.428331 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] PATH_INFO: `/v3/users/6719d09e7e9243cb90029d01d0c5aa92` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:07.429569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6719d09e7e9243cb90029d01d0c5aa92)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:07.432176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:07.611633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6719d09e7e9243cb90029d01d0c5aa92'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.735959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6719d09e7e9243cb90029d01d0c5aa92'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.749617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6719d09e7e9243cb90029d01d0c5aa92'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.750376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6719d09e7e9243cb90029d01d0c5aa92'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.750999 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6719d09e7e9243cb90029d01d0c5aa92'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:07.763586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1304/2593] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:07 2022] DELETE /identity/v3/users/6719d09e7e9243cb90029d01d0c5aa92 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:08.284854 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:08.285370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:08.285468 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:08.308879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:43:08.310042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:08.368706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:08.370422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:08.391027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1290/2594] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 02:43:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:43:08.397767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:08.410529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:08.410529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:08.462678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb079bfe-e1ff-4455-b0d3-712851362300 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:08.467356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:08.467980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:08.468254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:08.468529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] PATH_INFO: `/v3/projects/f8cfcc1b071f4a57a34cadef1135c8d7` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:08.469655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8cfcc1b071f4a57a34cadef1135c8d7)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:08.471653 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:08.484591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9c4d-a879-4803-9705-adc06427f356 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:08.490006 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:08.492715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:08.492715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:08.492715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:08.500798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:08.502410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:08.511645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1291/2595] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:43:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:43:08.602514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8cfcc1b071f4a57a34cadef1135c8d7', 'request_id': 'req-19157eaa-48c2-48ac-985b-50ef88ead8d6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:08.690920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8cfcc1b071f4a57a34cadef1135c8d7', 'request_id': 'req-19157eaa-48c2-48ac-985b-50ef88ead8d6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:08.705744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8cfcc1b071f4a57a34cadef1135c8d7', 'request_id': 'req-19157eaa-48c2-48ac-985b-50ef88ead8d6'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:08.722733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] The token cache is being invalidate because project f8cfcc1b071f4a57a34cadef1135c8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:08.728982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:08.728982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:08.728982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1305/2596] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:08 2022] DELETE /identity/v3/projects/f8cfcc1b071f4a57a34cadef1135c8d7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:13.195662 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:13.196590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:13.211168 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e51ca-096e-4f63-9fce-daf94476a9c0 service swift] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:13.231383 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:13.267134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:13.267681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:13.267954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:13.268223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] PATH_INFO: `/v3/users/0fe07f55f4d54059a193d4611e0a4f92` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:13.269321 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe07f55f4d54059a193d4611e0a4f92)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:13.271254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:13.394528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe07f55f4d54059a193d4611e0a4f92'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:13.449524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe07f55f4d54059a193d4611e0a4f92'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:13.462580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe07f55f4d54059a193d4611e0a4f92'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:13.469271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe07f55f4d54059a193d4611e0a4f92'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:13.469918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe07f55f4d54059a193d4611e0a4f92'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:13.475825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1292/2597] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:13 2022] DELETE /identity/v3/users/0fe07f55f4d54059a193d4611e0a4f92 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:43:13.691704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:13.709278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:13.709278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19157eaa-48c2-48ac-985b-50ef88ead8d6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:13.768827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:13.769494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:13.769746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:13.770015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] PATH_INFO: `/v3/projects/e069855c07d94296a331d49a7a2a7ce4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:13.771162 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e069855c07d94296a331d49a7a2a7ce4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:13.773034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:13.996527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:14.000235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e069855c07d94296a331d49a7a2a7ce4', 'request_id': 'req-73135198-a6da-4072-b684-df1f27b28e3c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.055401 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e069855c07d94296a331d49a7a2a7ce4', 'request_id': 'req-73135198-a6da-4072-b684-df1f27b28e3c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.062181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e069855c07d94296a331d49a7a2a7ce4', 'request_id': 'req-73135198-a6da-4072-b684-df1f27b28e3c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.073732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] The token cache is being invalidate because project e069855c07d94296a331d49a7a2a7ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:14.074026 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.074671 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.076539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1306/2598] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:13 2022] DELETE /identity/v3/projects/e069855c07d94296a331d49a7a2a7ce4 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:14.080667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:14.082050 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:14.095581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca62d7-bb7d-4f3d-9e2b-9124fd0cc24c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:14.153079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:14.153703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:14.153980 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:14.154252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] PATH_INFO: `/v3/users/4f791616148c4fa08c83c699b0a21077` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:14.155398 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f791616148c4fa08c83c699b0a21077)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:14.157336 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:14.359315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:14.360811 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f791616148c4fa08c83c699b0a21077'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.444045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f791616148c4fa08c83c699b0a21077'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.457265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f791616148c4fa08c83c699b0a21077'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.464449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f791616148c4fa08c83c699b0a21077'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.465118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f791616148c4fa08c83c699b0a21077'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.470871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1293/2599] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:43:14 2022] DELETE /identity/v3/users/4f791616148c4fa08c83c699b0a21077 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:14.714656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:14.716118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:14.729520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73135198-a6da-4072-b684-df1f27b28e3c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:14.786100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:14.786751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:14.787030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:14.787298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] PATH_INFO: `/v3/projects/cfe4d8665af44a05b457b11d810a13b4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:14.788443 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe4d8665af44a05b457b11d810a13b4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:14.790956 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:14.872307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe4d8665af44a05b457b11d810a13b4', 'request_id': 'req-b07048a2-651a-4402-a74e-100591e38fa1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.912682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe4d8665af44a05b457b11d810a13b4', 'request_id': 'req-b07048a2-651a-4402-a74e-100591e38fa1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.920925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe4d8665af44a05b457b11d810a13b4', 'request_id': 'req-b07048a2-651a-4402-a74e-100591e38fa1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.931928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] The token cache is being invalidate because project cfe4d8665af44a05b457b11d810a13b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:43:14.932216 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.932859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:43:14.934295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1307/2600] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:43:14 2022] DELETE /identity/v3/projects/cfe4d8665af44a05b457b11d810a13b4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:43:24.507394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=13,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:24.928532 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:10.502048 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:10.503711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:10.617761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa187d-30bf-4f46-ac0d-2c3c82dd0a18 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:10.880349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:10.880994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:10.881919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:10.882208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:10.884154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:10.899214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:10.900831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:10.912096 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1294/2601] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:15.321971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:15.323450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:15.380320 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07048a2-651a-4402-a74e-100591e38fa1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:15.383903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:15.384495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:15.384773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:15.385043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:15.386760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:15.449448 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:15.451140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:15.460761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1308/2602] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:19.271122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:19.272559 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:19.286800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48173de7-6353-444a-af8c-2c7c46636490 service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:19.291411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:19.292060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:19.292350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:19.292600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] PATH_INFO: `/v3/users/7fd9eb21aa634650a7022b7b4ca145fd` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:19.293696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd9eb21aa634650a7022b7b4ca145fd)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:19.296181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:19.479911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd9eb21aa634650a7022b7b4ca145fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:19.540065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd9eb21aa634650a7022b7b4ca145fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:19.552550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd9eb21aa634650a7022b7b4ca145fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:19.557701 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:19.559645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd9eb21aa634650a7022b7b4ca145fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:19.560313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-64a5ae57-cad5-4348-a677-70516e4ac32f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd9eb21aa634650a7022b7b4ca145fd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:19.566156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1295/2603] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:19 2022] DELETE /identity/v3/users/7fd9eb21aa634650a7022b7b4ca145fd => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:19.804602 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:19.816549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8a9b1-785b-44fc-9e9c-a6ef51c62282 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:19.875851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:19.876465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:19.876739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:19.877005 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] PATH_INFO: `/v3/projects/dddb64c31c0a4db2a184543c435c7bc0` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:19.878101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dddb64c31c0a4db2a184543c435c7bc0)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:19.880001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:19.963487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dddb64c31c0a4db2a184543c435c7bc0', 'request_id': 'req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:19.965938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:20.042944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dddb64c31c0a4db2a184543c435c7bc0', 'request_id': 'req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:20.049577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dddb64c31c0a4db2a184543c435c7bc0', 'request_id': 'req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:20.061577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] The token cache is being invalidate because project dddb64c31c0a4db2a184543c435c7bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:44:20.061577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:20.061577 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c0a44e2b-66dc-45dd-a414-b4b8ddbfca2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:20.063363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1309/2604] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:44:19 2022] DELETE /identity/v3/projects/dddb64c31c0a4db2a184543c435c7bc0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:20.068292 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101353be-f1eb-4866-ab22-447945d0f996 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.068582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101353be-f1eb-4866-ab22-447945d0f996 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.068857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101353be-f1eb-4866-ab22-447945d0f996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.089719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-101353be-f1eb-4866-ab22-447945d0f996 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:20.091020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-101353be-f1eb-4866-ab22-447945d0f996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:20.149915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1296/2605] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:20.158009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef90d40-bbbb-4808-8932-c23a52105ae6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.158295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef90d40-bbbb-4808-8932-c23a52105ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.159526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef90d40-bbbb-4808-8932-c23a52105ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.179474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2ef90d40-bbbb-4808-8932-c23a52105ae6 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:20.180696 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2ef90d40-bbbb-4808-8932-c23a52105ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:20.189433 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1310/2606] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:20.193644 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101353be-f1eb-4866-ab22-447945d0f996 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:20.204185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101353be-f1eb-4866-ab22-447945d0f996 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:20.209013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:20.209611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.209889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.210158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.210965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:44:20.211266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:20.212856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:20.224963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1297/2607] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:44:20.229546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef90d40-bbbb-4808-8932-c23a52105ae6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:20.239224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef90d40-bbbb-4808-8932-c23a52105ae6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:20.244210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:20.244792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.245073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.245343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.245962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:20.247882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:20.341472 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1311/2608] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:20 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:44:20.346007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:20.355991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f08e9c-731d-41f9-aa57-684ce2539907 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:20.412526 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:20.413216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.413496 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.413763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.414426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:20.416348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:20.506719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1298/2609] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:20 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:44:20.512087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:20.521133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fcb67-6f5b-463e-8fc7-cfc4011d6fd2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:20.525739 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:20.526312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.526617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.526884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.527512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:20.529063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:20.539315 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1312/2610] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:20.544759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:20.555771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5613f-2614-48f3-83a2-8d1e4778c712 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:20.560547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980eff3-8014-43ec-8a1b-6fc16367b9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:20.561239 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980eff3-8014-43ec-8a1b-6fc16367b9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.561516 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980eff3-8014-43ec-8a1b-6fc16367b9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.561807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980eff3-8014-43ec-8a1b-6fc16367b9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.562472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980eff3-8014-43ec-8a1b-6fc16367b9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:20.564038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980eff3-8014-43ec-8a1b-6fc16367b9d8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:20.574063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1299/2611] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:20.578673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:20.588900 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca7f28-4bde-47ec-828b-ba772a6a8720 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:20.594159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:20.594782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.595063 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.595339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] PATH_INFO: `/v3/projects/13640ce1c7ca4176978c4144862ff1ad/users/174fe89442b64ab0abadc7e2e5a9115a/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.611442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13640ce1c7ca4176978c4144862ff1ad, user_id=174fe89442b64ab0abadc7e2e5a9115a, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:20.614057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:20.665559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1313/2612] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:20 2022] PUT /identity/v3/projects/13640ce1c7ca4176978c4144862ff1ad/users/174fe89442b64ab0abadc7e2e5a9115a/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:20.677361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00530bea-e6db-4b1b-87d6-71665736a0a2 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.677666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00530bea-e6db-4b1b-87d6-71665736a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.682433 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00530bea-e6db-4b1b-87d6-71665736a0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.706076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-00530bea-e6db-4b1b-87d6-71665736a0a2 None None] MFA Rules not processed for user `174fe89442b64ab0abadc7e2e5a9115a`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:20.707318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-00530bea-e6db-4b1b-87d6-71665736a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:20.756208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:20.773341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1300/2613] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:20.785286 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:20.786995 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:20.864194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eec94f-b006-442e-b625-3f5efa292210 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:20.870359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9f317-7da9-424e-b445-0a7ad0544ee4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:20.872638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9f317-7da9-424e-b445-0a7ad0544ee4 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:20.872638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9f317-7da9-424e-b445-0a7ad0544ee4 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:20.872638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9f317-7da9-424e-b445-0a7ad0544ee4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:20.880682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9f317-7da9-424e-b445-0a7ad0544ee4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:20.882246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9f317-7da9-424e-b445-0a7ad0544ee4 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:20.891678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1314/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:30.837178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:30.887048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=3,keystone:SELECT=67,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:31.074229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00530bea-e6db-4b1b-87d6-71665736a0a2 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:31.136049 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00530bea-e6db-4b1b-87d6-71665736a0a2 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:31.140605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:31.141355 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:31.141665 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:31.142528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] PATH_INFO: `/v3/users/174fe89442b64ab0abadc7e2e5a9115a` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:31.143668 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] RBAC: Authorizing `identity:delete_user(user_id=174fe89442b64ab0abadc7e2e5a9115a)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:31.145574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:31.301683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174fe89442b64ab0abadc7e2e5a9115a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:31.368681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174fe89442b64ab0abadc7e2e5a9115a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:31.382234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174fe89442b64ab0abadc7e2e5a9115a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:31.389715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174fe89442b64ab0abadc7e2e5a9115a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:31.390527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174fe89442b64ab0abadc7e2e5a9115a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:31.396553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1301/2615] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:31 2022] DELETE /identity/v3/users/174fe89442b64ab0abadc7e2e5a9115a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:31.401192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:31.401479 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:31.401755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:31.420963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:31.422173 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:31.482508 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1315/2616] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:31.492717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:31.494149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:31.553243 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd4a7-3b9f-4a93-a235-169c9dea680a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:31.557038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb58f1-5fe3-4a5b-8fc6-9a0eb527e394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:31.557637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58f1-5fe3-4a5b-8fc6-9a0eb527e394 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:31.557912 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58f1-5fe3-4a5b-8fc6-9a0eb527e394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:31.558178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb58f1-5fe3-4a5b-8fc6-9a0eb527e394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:31.568523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb58f1-5fe3-4a5b-8fc6-9a0eb527e394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:31.569943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb58f1-5fe3-4a5b-8fc6-9a0eb527e394 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:31.579790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1302/2617] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:32.167529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:32.169410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:32.182189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ae132-edf0-4876-ac1d-fb49949b75c9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:32.187263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:32.187864 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.188135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.188400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] PATH_INFO: `/v3/projects/13640ce1c7ca4176978c4144862ff1ad` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.189712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13640ce1c7ca4176978c4144862ff1ad)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:32.191639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:32.277938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13640ce1c7ca4176978c4144862ff1ad', 'request_id': 'req-44cc4234-aa76-4622-a1ff-de6138408d39'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:32.319108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13640ce1c7ca4176978c4144862ff1ad', 'request_id': 'req-44cc4234-aa76-4622-a1ff-de6138408d39'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:32.326098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13640ce1c7ca4176978c4144862ff1ad', 'request_id': 'req-44cc4234-aa76-4622-a1ff-de6138408d39'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:32.339843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] The token cache is being invalidate because project 13640ce1c7ca4176978c4144862ff1ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:44:32.340138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:32.340818 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-44cc4234-aa76-4622-a1ff-de6138408d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:32.346496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1316/2618] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:44:32 2022] DELETE /identity/v3/projects/13640ce1c7ca4176978c4144862ff1ad => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:32.348407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c172d3-0851-4dd8-bcfc-dec100a75542 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.348858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c172d3-0851-4dd8-bcfc-dec100a75542 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.348972 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c172d3-0851-4dd8-bcfc-dec100a75542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.371694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-b8c172d3-0851-4dd8-bcfc-dec100a75542 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:32.372957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b8c172d3-0851-4dd8-bcfc-dec100a75542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:32.390530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:32.433311 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1303/2619] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:32.441689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee94a-514e-4018-a3b2-5e422a5094dc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.441976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee94a-514e-4018-a3b2-5e422a5094dc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.442254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee94a-514e-4018-a3b2-5e422a5094dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.462858 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5e9ee94a-514e-4018-a3b2-5e422a5094dc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:32.464108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5e9ee94a-514e-4018-a3b2-5e422a5094dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:32.472214 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1317/2620] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:32.476641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c172d3-0851-4dd8-bcfc-dec100a75542 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:32.486421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c172d3-0851-4dd8-bcfc-dec100a75542 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:32.491149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:32.491735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.492080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.492354 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.492990 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:44:32.493288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:32.494910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:32.511090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1304/2621] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:44:32.512385 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ee94a-514e-4018-a3b2-5e422a5094dc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:32.523043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ee94a-514e-4018-a3b2-5e422a5094dc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:32.527810 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:32.528490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.528839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.529119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.529753 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:32.531658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:32.617122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1318/2622] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:32 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:32.619348 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:32.630648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3fe0a-5cf3-4b59-b083-84dcf9d78c2f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:32.697551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:32.699276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.699276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.699276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.700564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:32.702538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:32.792960 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1305/2623] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:32 2022] POST /identity/v3/users => generated 303 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:32.797421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:32.815278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc64c2-4ac6-4a41-906e-a5a9380fc21b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:32.822010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:32.822805 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.823136 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.824786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.824786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:32.826219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:32.838287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1319/2624] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:32.842334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:32.853531 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4653cd-5342-475a-9a9d-03355f98d92d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:32.858820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997a998-b9a2-40c8-ac95-03c857119779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:32.859430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a998-b9a2-40c8-ac95-03c857119779 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.859828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a998-b9a2-40c8-ac95-03c857119779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.860093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a998-b9a2-40c8-ac95-03c857119779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.862010 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997a998-b9a2-40c8-ac95-03c857119779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:32.863683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997a998-b9a2-40c8-ac95-03c857119779 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:32.879129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1306/2625] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:44:32.880109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:32.891827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d640f66-4ecb-4f35-ae58-a33a0244de6a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:32.896601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:32.897257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.897534 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.897802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] PATH_INFO: `/v3/projects/2a3f2e39693345d19aa170f2ce488ced/users/6952ed9f20384fa4bf6fcefb8cd267be/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:32.919193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a3f2e39693345d19aa170f2ce488ced, user_id=6952ed9f20384fa4bf6fcefb8cd267be, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:32.922040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:32.980768 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1320/2626] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:32 2022] PUT /identity/v3/projects/2a3f2e39693345d19aa170f2ce488ced/users/6952ed9f20384fa4bf6fcefb8cd267be/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:32.993291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865a04-c64f-457b-861d-70ca80ca1a15 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:32.993580 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865a04-c64f-457b-861d-70ca80ca1a15 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:32.993851 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865a04-c64f-457b-861d-70ca80ca1a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.021869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-2c865a04-c64f-457b-861d-70ca80ca1a15 None None] MFA Rules not processed for user `6952ed9f20384fa4bf6fcefb8cd267be`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:33.023230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-2c865a04-c64f-457b-861d-70ca80ca1a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:33.099396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1307/2627] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:44:33.103728 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.127365 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:33.171829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccaafc-22c8-4f25-aaf8-e1577489b1bc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.179477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.180177 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.180473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.181158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.181509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.183589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:33.286154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1321/2628] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:33 2022] POST /identity/v3/projects => generated 405 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:44:33.290042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c865a04-c64f-457b-861d-70ca80ca1a15 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.301992 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c865a04-c64f-457b-861d-70ca80ca1a15 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.316514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:33.363114 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.363734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.364179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.364370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.364935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.367107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:33.484700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1308/2629] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:33 2022] POST /identity/v3/users => generated 309 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:44:33.489208 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.503447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1042ce-8288-4480-b006-1c427255e26c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.506335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.507016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.507296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.507563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.508207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.509798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:33.521959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1322/2630] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:44:33.525796 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.536770 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba090f-ad71-42f1-b18c-2f21f565480b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.541618 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.542277 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.542590 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.542880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] PATH_INFO: `/v3/projects/af6abd5b1b0947da97db5da56974df09/users/7015125baffa4464800e780b3f0f2c13/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.560548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6abd5b1b0947da97db5da56974df09, user_id=7015125baffa4464800e780b3f0f2c13, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.562610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:33.654768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1309/2631] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:33 2022] PUT /identity/v3/projects/af6abd5b1b0947da97db5da56974df09/users/7015125baffa4464800e780b3f0f2c13/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:33.659404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.670469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21dda2-6c57-4394-92a2-bad03bf6723c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.735123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aee545-0960-4504-8454-bcba2fda7699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.736156 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aee545-0960-4504-8454-bcba2fda7699 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.736662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aee545-0960-4504-8454-bcba2fda7699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.737096 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aee545-0960-4504-8454-bcba2fda7699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.738010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aee545-0960-4504-8454-bcba2fda7699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.742593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aee545-0960-4504-8454-bcba2fda7699 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:33.756099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1323/2632] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:33.760560 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.775270 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c1efb-1eb2-458a-b944-a540298a8a28 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.780606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.781265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.781432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.781704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] PATH_INFO: `/v3/projects/af6abd5b1b0947da97db5da56974df09/users/7015125baffa4464800e780b3f0f2c13/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.784631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6abd5b1b0947da97db5da56974df09, user_id=7015125baffa4464800e780b3f0f2c13, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.787294 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:33.799039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1310/2633] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:33 2022] PUT /identity/v3/projects/af6abd5b1b0947da97db5da56974df09/users/7015125baffa4464800e780b3f0f2c13/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:33.809425 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a21a9-7cc4-4bb7-9b23-f1f1e0c5dd10 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.809710 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a21a9-7cc4-4bb7-9b23-f1f1e0c5dd10 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.809981 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a21a9-7cc4-4bb7-9b23-f1f1e0c5dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.838982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3e9a21a9-7cc4-4bb7-9b23-f1f1e0c5dd10 None None] MFA Rules not processed for user `7015125baffa4464800e780b3f0f2c13`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:33.840206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3e9a21a9-7cc4-4bb7-9b23-f1f1e0c5dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:33.911772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1324/2634] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:33.922622 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:33.925557 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:33.982297 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ededb0-cad4-4643-abbb-5402cb9abd12 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:33.986218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508058b-a6dd-42da-8577-49ce75b11c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:33.986833 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508058b-a6dd-42da-8577-49ce75b11c02 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:33.987134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508058b-a6dd-42da-8577-49ce75b11c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:33.987401 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508058b-a6dd-42da-8577-49ce75b11c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:33.996169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508058b-a6dd-42da-8577-49ce75b11c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:33.997562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508058b-a6dd-42da-8577-49ce75b11c02 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:34.007573 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1311/2635] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:33 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:34.138535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a21a9-7cc4-4bb7-9b23-f1f1e0c5dd10 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:34.156100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:34.199928 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a21a9-7cc4-4bb7-9b23-f1f1e0c5dd10 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:34.205734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:34.206345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:34.206652 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:34.206915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] PATH_INFO: `/v3/users/6952ed9f20384fa4bf6fcefb8cd267be` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:34.208018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6952ed9f20384fa4bf6fcefb8cd267be)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:34.209890 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:34.344256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6952ed9f20384fa4bf6fcefb8cd267be'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:34.391537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6952ed9f20384fa4bf6fcefb8cd267be'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:34.405254 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6952ed9f20384fa4bf6fcefb8cd267be'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:34.406051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6952ed9f20384fa4bf6fcefb8cd267be'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:34.406873 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6952ed9f20384fa4bf6fcefb8cd267be'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:34.427395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1325/2636] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:34 2022] DELETE /identity/v3/users/6952ed9f20384fa4bf6fcefb8cd267be => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:34.429842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:34.430374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:34.430828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:34.450963 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:34.452116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:34.512615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1312/2637] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:44:34.522969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:34.524388 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:34.587016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d22ae-ddc4-4f0e-846c-725b5c834a4e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:34.594291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:34.599490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:34.599490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:34.599490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:34.607028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:34.608455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:34.619249 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1326/2638] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:35.166022 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:35.168490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:35.183439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9767f-affe-4f04-93c9-9698bf1d3c60 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:35.191795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:35.192362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:35.192825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:35.193288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] PATH_INFO: `/v3/projects/2a3f2e39693345d19aa170f2ce488ced` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:35.195924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a3f2e39693345d19aa170f2ce488ced)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:35.200080 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:35.218713 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:35.336720 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a3f2e39693345d19aa170f2ce488ced', 'request_id': 'req-22aa9f3a-5564-440b-9da8-baaaba9b3093'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.343396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a3f2e39693345d19aa170f2ce488ced', 'request_id': 'req-22aa9f3a-5564-440b-9da8-baaaba9b3093'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.431801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a3f2e39693345d19aa170f2ce488ced', 'request_id': 'req-22aa9f3a-5564-440b-9da8-baaaba9b3093'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.450410 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] The token cache is being invalidate because project 2a3f2e39693345d19aa170f2ce488ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:44:35.450711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.452436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.456078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1313/2639] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:44:35 2022] DELETE /identity/v3/projects/2a3f2e39693345d19aa170f2ce488ced => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:35.462488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:35.472716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e496d42-f235-41c0-937f-da341dce3eae service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:35.507867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:35.531785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:35.532685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:35.533106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:35.533381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] PATH_INFO: `/v3/users/7015125baffa4464800e780b3f0f2c13` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:35.534578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7015125baffa4464800e780b3f0f2c13)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:35.536482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:35.698271 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7015125baffa4464800e780b3f0f2c13'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.752221 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7015125baffa4464800e780b3f0f2c13'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.765626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7015125baffa4464800e780b3f0f2c13'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.766332 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7015125baffa4464800e780b3f0f2c13'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.767638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4bbb5008-06d8-4f08-b251-2ccfecafe8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7015125baffa4464800e780b3f0f2c13'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:35.780391 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1327/2640] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:35 2022] DELETE /identity/v3/users/7015125baffa4464800e780b3f0f2c13 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:44:36.169746 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:36.171325 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:36.186744 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9f3a-5564-440b-9da8-baaaba9b3093 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:36.247012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:36.247607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.247881 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.248148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] PATH_INFO: `/v3/projects/af6abd5b1b0947da97db5da56974df09` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.249280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6abd5b1b0947da97db5da56974df09)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:36.251275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:36.369274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6abd5b1b0947da97db5da56974df09', 'request_id': 'req-1fae3f0c-a11d-4116-b1b5-5004c2da232a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:36.375384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6abd5b1b0947da97db5da56974df09', 'request_id': 'req-1fae3f0c-a11d-4116-b1b5-5004c2da232a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:36.454582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6abd5b1b0947da97db5da56974df09', 'request_id': 'req-1fae3f0c-a11d-4116-b1b5-5004c2da232a'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:36.465278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] The token cache is being invalidate because project af6abd5b1b0947da97db5da56974df09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:44:36.465568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:36.466187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1fae3f0c-a11d-4116-b1b5-5004c2da232a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:36.467678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1314/2641] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:44:36 2022] DELETE /identity/v3/projects/af6abd5b1b0947da97db5da56974df09 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:36.472484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ac09a-e58d-4eb2-b223-c32143f24f77 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.472934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ac09a-e58d-4eb2-b223-c32143f24f77 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.473049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ac09a-e58d-4eb2-b223-c32143f24f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.492815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-969ac09a-e58d-4eb2-b223-c32143f24f77 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:36.494010 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-969ac09a-e58d-4eb2-b223-c32143f24f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:36.555185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1328/2642] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:36.562930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66427-ea77-4ec7-8e33-91621590b036 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.563216 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66427-ea77-4ec7-8e33-91621590b036 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.564063 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b66427-ea77-4ec7-8e33-91621590b036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.584007 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-68b66427-ea77-4ec7-8e33-91621590b036 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:36.585240 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-68b66427-ea77-4ec7-8e33-91621590b036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:36.593406 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1315/2643] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:36.597819 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ac09a-e58d-4eb2-b223-c32143f24f77 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:36.607606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ac09a-e58d-4eb2-b223-c32143f24f77 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:36.612888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:36.613469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.613742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.614014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.614681 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:44:36.614982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:36.616549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:36.631940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1329/2644] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:44:36.635872 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b66427-ea77-4ec7-8e33-91621590b036 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:36.642931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b66427-ea77-4ec7-8e33-91621590b036 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:36.647567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:36.648155 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.648428 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.648692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.649307 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:36.651225 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:36.746470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:36.758118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1316/2645] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:36 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:36.759197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:36.769891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0c8c-dd06-45cd-8a78-38e1f07bfc18 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:36.814000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:36.829344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:36.833023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.833023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.833023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.833023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:36.833394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:36.911329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1330/2646] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:36 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:44:36.915541 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:36.927125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c763e1-d4e1-4f7f-a6b1-17092b2bbca1 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:36.931672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:36.933807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.933807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.933807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.933807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:36.935015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:36.945102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1317/2647] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:36.949191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:36.959343 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ed0e1-e492-46b0-b198-693800591200 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:36.964080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d61e61-5850-4506-ad11-5c1fe0647263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:36.964669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d61e61-5850-4506-ad11-5c1fe0647263 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.964943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d61e61-5850-4506-ad11-5c1fe0647263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:36.965209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d61e61-5850-4506-ad11-5c1fe0647263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:36.965844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d61e61-5850-4506-ad11-5c1fe0647263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:36.967503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d61e61-5850-4506-ad11-5c1fe0647263 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:36.979009 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1331/2648] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:36.983441 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:36.994318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed2c8-9db5-4b4a-9ffc-ce0f21eedb10 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:36.998953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:36.999629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:36.999893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.000156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] PATH_INFO: `/v3/projects/86e8e1a37751486fa8771d647328fa93/users/18f1360e66834b9694dc2ae65c1b8d56/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.014953 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e8e1a37751486fa8771d647328fa93, user_id=18f1360e66834b9694dc2ae65c1b8d56, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.017812 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.082914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1318/2649] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:36 2022] PUT /identity/v3/projects/86e8e1a37751486fa8771d647328fa93/users/18f1360e66834b9694dc2ae65c1b8d56/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:37.094279 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61488e-e4e8-4ccf-bcf1-52e4427a115f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.095167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61488e-e4e8-4ccf-bcf1-52e4427a115f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.095442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61488e-e4e8-4ccf-bcf1-52e4427a115f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.124817 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1f61488e-e4e8-4ccf-bcf1-52e4427a115f None None] MFA Rules not processed for user `18f1360e66834b9694dc2ae65c1b8d56`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:37.126116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1f61488e-e4e8-4ccf-bcf1-52e4427a115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:37.192083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1332/2650] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:37.196700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:37.257471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12765f-622b-4bb7-af62-d9af41304abe admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:37.262244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:37.262929 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.263209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.263476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.264086 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.266016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.323585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1319/2651] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:37 2022] POST /identity/v3/projects => generated 359 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:37.327903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61488e-e4e8-4ccf-bcf1-52e4427a115f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:37.339813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61488e-e4e8-4ccf-bcf1-52e4427a115f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:37.397411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:37.397998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.398267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.398562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.399179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.401187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.465565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1333/2652] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:37 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:37.470316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:37.480777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d993-fd3d-447f-9846-91e62acd257e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:37.485431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:37.486015 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.486288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.486611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.487262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.488829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.499037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1320/2653] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:37.503550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:37.513462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05854bd-42d8-4277-9758-078cec7cea50 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:37.518816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:37.519386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.519659 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.519984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] PATH_INFO: `/v3/projects/657fde3570b643b6a6500095281d482d/users/876aa17ecbba4bf0ba6793a4ad9847b4/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.535179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=657fde3570b643b6a6500095281d482d, user_id=876aa17ecbba4bf0ba6793a4ad9847b4, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.537764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.607868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1334/2654] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:37 2022] PUT /identity/v3/projects/657fde3570b643b6a6500095281d482d/users/876aa17ecbba4bf0ba6793a4ad9847b4/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:37.612045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:37.622031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1c587-5153-4402-8903-cc21be1b1c6a admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:37.676850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7e721-6d3d-4a55-bcae-1d5e96dbc069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:37.677438 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7e721-6d3d-4a55-bcae-1d5e96dbc069 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.677710 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7e721-6d3d-4a55-bcae-1d5e96dbc069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.677974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7e721-6d3d-4a55-bcae-1d5e96dbc069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.678666 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7e721-6d3d-4a55-bcae-1d5e96dbc069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.680229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7e721-6d3d-4a55-bcae-1d5e96dbc069 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.690409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1321/2655] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:37.694441 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:37.705622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a339d92-5142-498d-aa1d-910db841fe3f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:37.710203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:37.710854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.711129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.711396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] PATH_INFO: `/v3/projects/657fde3570b643b6a6500095281d482d/users/876aa17ecbba4bf0ba6793a4ad9847b4/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.713281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657fde3570b643b6a6500095281d482d, user_id=876aa17ecbba4bf0ba6793a4ad9847b4, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.715907 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.725935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1335/2656] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:37 2022] PUT /identity/v3/projects/657fde3570b643b6a6500095281d482d/users/876aa17ecbba4bf0ba6793a4ad9847b4/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:37.735905 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed261a-9dd7-4f48-8747-82f4696eb825 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.736261 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed261a-9dd7-4f48-8747-82f4696eb825 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.736538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed261a-9dd7-4f48-8747-82f4696eb825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.764400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-efed261a-9dd7-4f48-8747-82f4696eb825 None None] MFA Rules not processed for user `876aa17ecbba4bf0ba6793a4ad9847b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:37.765619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-efed261a-9dd7-4f48-8747-82f4696eb825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:37.785535 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:37.835556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1322/2657] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:37 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:37.847149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:37.848807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:37.859960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:37.918606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be4ae-c7e5-4df8-a846-4e4fe2fe9ce0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:37.918606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:37.918606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:37.918606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:37.923384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:37.931869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:37.933318 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:37.943376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1336/2658] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:37 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:38.127037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed261a-9dd7-4f48-8747-82f4696eb825 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:38.136664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed261a-9dd7-4f48-8747-82f4696eb825 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:38.141046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:38.142395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:38.142676 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:38.142941 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:38.201521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:38.202999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:38.211823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1323/2659] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:39.244769 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:39.246183 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:39.303312 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329ed1-ee5a-4da6-9185-8c86269b039d service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:39.306970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:39.307560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:39.307839 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:39.308108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:39.309799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:39.370705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:39.372109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:39.382121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1337/2660] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:42.980890 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:42.982402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:42.997450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322595-3472-44dc-85b3-c6e50cc54063 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:43.002481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:43.003091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:43.003361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:43.003628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] PATH_INFO: `/v3/users/55afeedd651248da8ac80c61fb0729cc` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:43.004735 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55afeedd651248da8ac80c61fb0729cc)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:43.006692 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:43.206957 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55afeedd651248da8ac80c61fb0729cc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.254287 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55afeedd651248da8ac80c61fb0729cc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.266686 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55afeedd651248da8ac80c61fb0729cc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.274409 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55afeedd651248da8ac80c61fb0729cc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.275111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a23b1c79-a56a-416c-b4d9-355896bbc936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55afeedd651248da8ac80c61fb0729cc'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.280754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1324/2661] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:42 2022] DELETE /identity/v3/users/55afeedd651248da8ac80c61fb0729cc => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:43.620943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:43.632274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ee1d3-72c1-42eb-800e-b95d769f8e41 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:43.685870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:43.691980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:43.692588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:43.692860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:43.693127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] PATH_INFO: `/v3/projects/897030a5075b4ca3ad29d73371306a66` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:43.694227 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] RBAC: Authorizing `identity:delete_project(project_id=897030a5075b4ca3ad29d73371306a66)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:43.696137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:43.815087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897030a5075b4ca3ad29d73371306a66', 'request_id': 'req-868f1026-5d54-4db7-888d-4aafe624659e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.882295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897030a5075b4ca3ad29d73371306a66', 'request_id': 'req-868f1026-5d54-4db7-888d-4aafe624659e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.888775 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897030a5075b4ca3ad29d73371306a66', 'request_id': 'req-868f1026-5d54-4db7-888d-4aafe624659e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.899155 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] The token cache is being invalidate because project 897030a5075b4ca3ad29d73371306a66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:44:43.899440 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.900074 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-868f1026-5d54-4db7-888d-4aafe624659e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:43.901539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1338/2662] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:44:43 2022] DELETE /identity/v3/projects/897030a5075b4ca3ad29d73371306a66 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:43.906393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48856e4a-5002-4eb9-8307-7117917aa76b None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:43.906681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48856e4a-5002-4eb9-8307-7117917aa76b None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:43.906948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48856e4a-5002-4eb9-8307-7117917aa76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:43.928279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-48856e4a-5002-4eb9-8307-7117917aa76b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:43.929489 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-48856e4a-5002-4eb9-8307-7117917aa76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:43.953220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:43.988412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1325/2663] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:43.996224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbac04-9bf5-46e7-9619-da943b0ef765 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:43.996506 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbac04-9bf5-46e7-9619-da943b0ef765 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:43.996778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbac04-9bf5-46e7-9619-da943b0ef765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.016376 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a4fbac04-9bf5-46e7-9619-da943b0ef765 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:44.018323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a4fbac04-9bf5-46e7-9619-da943b0ef765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:44.029974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1339/2664] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:44.035415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48856e4a-5002-4eb9-8307-7117917aa76b None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:44.045454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48856e4a-5002-4eb9-8307-7117917aa76b None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:44.050344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:44.050958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.051233 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.051502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.052139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:44:44.052435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:44.054020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:44.065411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1326/2665] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:44:44.069616 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbac04-9bf5-46e7-9619-da943b0ef765 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:44.078991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbac04-9bf5-46e7-9619-da943b0ef765 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:44.084558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:44.085130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.085402 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.085668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.086288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:44.088164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:44.159121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1340/2666] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:44 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:44.163587 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:44.173706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0c4b9-2a51-4f41-9d44-3742c1900fb3 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:44.229353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:44.229944 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.230223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.230523 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.231127 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:44.233024 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:44.339830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1327/2667] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:44 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:44.343881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:44.355360 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc1760-ec16-4d43-a091-2bd35cf4c5d4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:44.359891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:44.360519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.360791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.361624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.362256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:44.364870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:44.373942 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1341/2668] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:44.378119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:44.388020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15f17b-3248-4a31-95ed-806616eeede2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:44.393278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35abeae-453e-494d-9127-a51497d116a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:44.395251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35abeae-453e-494d-9127-a51497d116a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.395251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35abeae-453e-494d-9127-a51497d116a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.395251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35abeae-453e-494d-9127-a51497d116a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.395251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35abeae-453e-494d-9127-a51497d116a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:44.396748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35abeae-453e-494d-9127-a51497d116a6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:44.407187 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1328/2669] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:44.412085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:44.421915 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dd1fa-b411-4da3-a0d1-73ee5239adae admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:44.427258 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:44.427882 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.428154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.428422 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] PATH_INFO: `/v3/projects/570f253b2653418dabc6566536e14353/users/f3a60f8c44524cc4a7a394060e619b9d/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.443452 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] RBAC: Authorizing `identity:create_grant(project_id=570f253b2653418dabc6566536e14353, user_id=f3a60f8c44524cc4a7a394060e619b9d, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:44.446047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:44.540032 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1342/2670] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:44 2022] PUT /identity/v3/projects/570f253b2653418dabc6566536e14353/users/f3a60f8c44524cc4a7a394060e619b9d/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:44.545125 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.545717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.545717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.565252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:44.566395 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:44.626085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1329/2671] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:44.636235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:44.637964 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:44.695025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308fb8f-17aa-4137-bd39-5790c163274e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:44.698585 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3e4da-9529-4662-8859-ed42de1e627f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:44.699429 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e4da-9529-4662-8859-ed42de1e627f service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:44.699704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e4da-9529-4662-8859-ed42de1e627f service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:44.699970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e4da-9529-4662-8859-ed42de1e627f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:44.709626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3e4da-9529-4662-8859-ed42de1e627f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:44.711061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3e4da-9529-4662-8859-ed42de1e627f service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:44.720264 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1343/2672] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:50.065683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:50.067791 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:50.088800 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f1003-4285-4602-a97b-cbbcb3544214 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:50.094072 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:50.094766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:50.095060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:50.096070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] PATH_INFO: `/v3/users/18f1360e66834b9694dc2ae65c1b8d56` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:50.097616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f1360e66834b9694dc2ae65c1b8d56)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:50.099754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:50.209731 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:50.225306 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f1360e66834b9694dc2ae65c1b8d56'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:50.245744 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:50.246228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:50.246580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:50.278324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] MFA Rules not processed for user `f3a60f8c44524cc4a7a394060e619b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:50.279566 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:50.284615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f1360e66834b9694dc2ae65c1b8d56'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:50.295808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:50.314965 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f1360e66834b9694dc2ae65c1b8d56'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:50.331060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f1360e66834b9694dc2ae65c1b8d56'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:50.331817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-3bccd336-88a1-461a-bc77-2508aec5a05f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f1360e66834b9694dc2ae65c1b8d56'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:50.347843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1330/2673] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:50 2022] DELETE /identity/v3/users/18f1360e66834b9694dc2ae65c1b8d56 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:50.347843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:50.347843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:50.347843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:50.365931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:50.367739 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:50.381291 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1344/2674] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:44:50.389061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:50.392675 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:50.434879 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1331/2675] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:50.447667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:50.449567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:50.493123 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cb94f-5952-4ebf-9484-8fb9c3c275e7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:50.521223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef52df-ced2-46bf-920c-44a91da3d619 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:50.525252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:50.525864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:50.526139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:50.526444 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:50.537623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:50.539167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:50.551430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1332/2676] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:44:50.571709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:50.572373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:50.572655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:50.572932 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:50.638288 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:50.639903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:50.650313 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1345/2677] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:50.683027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:50.952988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:50.954678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:50.972706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144b57-9e2c-4226-b253-c7ae9efdfff9 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:50.977785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:50.978901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:50.983140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:50.983140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] PATH_INFO: `/v3/projects/86e8e1a37751486fa8771d647328fa93` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:50.983140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e8e1a37751486fa8771d647328fa93)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:50.983859 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:51.108999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:51.113841 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e8e1a37751486fa8771d647328fa93', 'request_id': 'req-b69cc3b0-b797-4ba1-9a5c-68a21206efac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.122020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e8e1a37751486fa8771d647328fa93', 'request_id': 'req-b69cc3b0-b797-4ba1-9a5c-68a21206efac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.186470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e8e1a37751486fa8771d647328fa93', 'request_id': 'req-b69cc3b0-b797-4ba1-9a5c-68a21206efac'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.199451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] The token cache is being invalidate because project 86e8e1a37751486fa8771d647328fa93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:44:51.199761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.200381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.201917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1333/2678] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:44:50 2022] DELETE /identity/v3/projects/86e8e1a37751486fa8771d647328fa93 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:51.206850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:51.217667 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0804c9-8078-4af9-9df4-152cb2bc1030 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:51.288715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:51.288715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:51.288715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:51.288715 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] PATH_INFO: `/v3/users/876aa17ecbba4bf0ba6793a4ad9847b4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:51.289668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] RBAC: Authorizing `identity:delete_user(user_id=876aa17ecbba4bf0ba6793a4ad9847b4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:51.292569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:51.547804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876aa17ecbba4bf0ba6793a4ad9847b4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.671481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876aa17ecbba4bf0ba6793a4ad9847b4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.684706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876aa17ecbba4bf0ba6793a4ad9847b4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.685413 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876aa17ecbba4bf0ba6793a4ad9847b4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.686028 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df8b9db-ae2c-4a0d-be0a-f7c64c3b377d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876aa17ecbba4bf0ba6793a4ad9847b4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:51.702237 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1346/2679] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:51 2022] DELETE /identity/v3/users/876aa17ecbba4bf0ba6793a4ad9847b4 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:52.153065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:52.154884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:52.183318 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cc3b0-b797-4ba1-9a5c-68a21206efac admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:52.249315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:52.250161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.250642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.250915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] PATH_INFO: `/v3/projects/657fde3570b643b6a6500095281d482d` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.252807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=657fde3570b643b6a6500095281d482d)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:52.254945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:52.352449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657fde3570b643b6a6500095281d482d', 'request_id': 'req-52c1494e-6140-44d5-a74a-0979ac80edbd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:52.359013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657fde3570b643b6a6500095281d482d', 'request_id': 'req-52c1494e-6140-44d5-a74a-0979ac80edbd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:52.403344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657fde3570b643b6a6500095281d482d', 'request_id': 'req-52c1494e-6140-44d5-a74a-0979ac80edbd'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:52.420423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] The token cache is being invalidate because project 657fde3570b643b6a6500095281d482d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:44:52.420423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:52.420423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-52c1494e-6140-44d5-a74a-0979ac80edbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:52.420423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1334/2680] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:44:52 2022] DELETE /identity/v3/projects/657fde3570b643b6a6500095281d482d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:52.424881 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0864d0-a41a-4efa-bd12-e37c790b945b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.425415 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0864d0-a41a-4efa-bd12-e37c790b945b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.425514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0864d0-a41a-4efa-bd12-e37c790b945b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.450298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7c0864d0-a41a-4efa-bd12-e37c790b945b None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:52.454606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7c0864d0-a41a-4efa-bd12-e37c790b945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:52.526712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1347/2681] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:52.535229 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3189ba3-1ae9-442f-ac63-e12235e2f358 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.535517 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3189ba3-1ae9-442f-ac63-e12235e2f358 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.535790 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3189ba3-1ae9-442f-ac63-e12235e2f358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.560829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e3189ba3-1ae9-442f-ac63-e12235e2f358 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:52.562044 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e3189ba3-1ae9-442f-ac63-e12235e2f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:52.571013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1335/2682] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:52.575773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0864d0-a41a-4efa-bd12-e37c790b945b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:52.586325 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0864d0-a41a-4efa-bd12-e37c790b945b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:52.595503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:52.598043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.598043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.598043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.598043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:44:52.598043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:52.599626 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:52.607210 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:52.618048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1348/2683] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:44:52.620331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3189ba3-1ae9-442f-ac63-e12235e2f358 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:52.632620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3189ba3-1ae9-442f-ac63-e12235e2f358 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:52.639655 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:52.640423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.640702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.642617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.643446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:52.645564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:52.702568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1336/2684] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:52 2022] POST /identity/v3/projects => generated 369 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:44:52.707068 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:52.718628 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7ea48-2328-493b-8781-14acde040d16 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:52.784356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:52.785327 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.785763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.786199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.787062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:52.790909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:52.860668 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1349/2685] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:52 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:52.865149 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:52.881302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a76cf-e836-4189-b6cf-89f3c49a4962 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:52.886694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:52.887376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.887847 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.888301 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.889220 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:52.890959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:52.909347 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:52.912295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1337/2686] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:44:52.945851 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e974b-fdea-4623-89e2-1b7268f3bae3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:52.951952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22fad2-44c7-4ccb-bb8f-c39aad4fa2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:52.952772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22fad2-44c7-4ccb-bb8f-c39aad4fa2db admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.953236 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22fad2-44c7-4ccb-bb8f-c39aad4fa2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.953965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22fad2-44c7-4ccb-bb8f-c39aad4fa2db admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:52.956202 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22fad2-44c7-4ccb-bb8f-c39aad4fa2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:52.958574 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22fad2-44c7-4ccb-bb8f-c39aad4fa2db admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:52.970902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1350/2687] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:52.975227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:52.985256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af41fa-2a68-43a6-8477-59fabff6dcce admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:52.994775 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0384ca1-1b48-44ae-bb6d-9a39535b527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:52.995506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0384ca1-1b48-44ae-bb6d-9a39535b527b admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:52.995921 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0384ca1-1b48-44ae-bb6d-9a39535b527b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:52.996201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0384ca1-1b48-44ae-bb6d-9a39535b527b admin admin] PATH_INFO: `/v3/projects/82c0466829d94e0d8bf039ad0a8df2c2/users/92db207d02614e1f820d812980366de1/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.014195 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0384ca1-1b48-44ae-bb6d-9a39535b527b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c0466829d94e0d8bf039ad0a8df2c2, user_id=92db207d02614e1f820d812980366de1, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:53.016249 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0384ca1-1b48-44ae-bb6d-9a39535b527b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:53.101435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:53.107169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1338/2688] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:52 2022] PUT /identity/v3/projects/82c0466829d94e0d8bf039ad0a8df2c2/users/92db207d02614e1f820d812980366de1/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:53.118632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117151-2712-4609-8256-9497b18b51b0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.118919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117151-2712-4609-8256-9497b18b51b0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.120118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34117151-2712-4609-8256-9497b18b51b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.163633 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-34117151-2712-4609-8256-9497b18b51b0 None None] MFA Rules not processed for user `92db207d02614e1f820d812980366de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:53.165356 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-34117151-2712-4609-8256-9497b18b51b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:53.236959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ad06a-b2ec-460a-bb33-5b7c5c0ef39c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.237484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ad06a-b2ec-460a-bb33-5b7c5c0ef39c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.237484 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ad06a-b2ec-460a-bb33-5b7c5c0ef39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.250889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1351/2689] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:53.255643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34117151-2712-4609-8256-9497b18b51b0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:53.261230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-077ad06a-b2ec-460a-bb33-5b7c5c0ef39c None None] MFA Rules not processed for user `f3a60f8c44524cc4a7a394060e619b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:53.262792 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-077ad06a-b2ec-460a-bb33-5b7c5c0ef39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:53.351385 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1339/2690] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:53.351503 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34117151-2712-4609-8256-9497b18b51b0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:53.354913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:53.355598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.355936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.356275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.357072 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:53.359482 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:53.374976 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:53.475833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1352/2691] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:44:53 2022] POST /identity/v3/projects => generated 369 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:44:53.480016 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ad06a-b2ec-460a-bb33-5b7c5c0ef39c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:53.492331 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ad06a-b2ec-460a-bb33-5b7c5c0ef39c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:53.562244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:53.562951 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.563228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.563490 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.565065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:53.567471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:53.655471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1340/2692] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:44:53 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:44:53.660423 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:53.674027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd55f33-b1d0-4187-85c5-af646116730c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:53.693838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:53.694520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.694798 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.695065 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.696396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:53.697294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:53.719837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1353/2693] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:53.724156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:53.734579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b403ef-56bb-4db6-9bf6-5bea5e5f5c44 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:53.740890 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:53.741658 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.742373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.742722 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] PATH_INFO: `/v3/projects/3d6465b0a8c54125aecaed0fe16287cf/users/cc6239b2d2c74f83bfe7f45adc7e5e48/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.763058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6465b0a8c54125aecaed0fe16287cf, user_id=cc6239b2d2c74f83bfe7f45adc7e5e48, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:53.766211 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:53.852781 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1341/2694] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:53 2022] PUT /identity/v3/projects/3d6465b0a8c54125aecaed0fe16287cf/users/cc6239b2d2c74f83bfe7f45adc7e5e48/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:44:53.853983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:53.865064 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470341c1-9ea1-4864-a66a-7fa963016df8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:53.931908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e48976-1432-4c4f-9f72-9aa15638ae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:53.932641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e48976-1432-4c4f-9f72-9aa15638ae59 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.932792 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e48976-1432-4c4f-9f72-9aa15638ae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.933060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e48976-1432-4c4f-9f72-9aa15638ae59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.933697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e48976-1432-4c4f-9f72-9aa15638ae59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:53.935305 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e48976-1432-4c4f-9f72-9aa15638ae59 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:53.946855 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1354/2695] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:44:53.951514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:53.962900 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d4ea0-1b98-4b8c-8bf3-dbe71918a024 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:53.967690 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:53.968282 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.968549 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.969174 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] PATH_INFO: `/v3/projects/3d6465b0a8c54125aecaed0fe16287cf/users/cc6239b2d2c74f83bfe7f45adc7e5e48/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:53.971338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6465b0a8c54125aecaed0fe16287cf, user_id=cc6239b2d2c74f83bfe7f45adc7e5e48, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:53.974706 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:53.985547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1342/2696] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:44:53 2022] PUT /identity/v3/projects/3d6465b0a8c54125aecaed0fe16287cf/users/cc6239b2d2c74f83bfe7f45adc7e5e48/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:53.996299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:53.996587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:53.996840 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:54.035301 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] MFA Rules not processed for user `cc6239b2d2c74f83bfe7f45adc7e5e48`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:54.036883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:54.110892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1355/2697] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:54.123135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:54.125087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:54.176224 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:54.178300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:54.193402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5a1e-8d7e-43e3-8c22-b82e314a5e81 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:54.197238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:54.197916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:54.198265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:54.198627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:54.210730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:54.212273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:54.218506 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:54.236379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1343/2698] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:54.254187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:44:54.259730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12967edb-aef1-4aef-84d5-3aaca42652d2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:54.265008 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:54.270638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:54.270638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:54.270638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:54.270638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:54.280261 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:54.281835 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:54.291791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1356/2699] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:54.378977 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:54.380474 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:54.393152 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba2ca7-0768-4354-bfe8-0f740e395aab service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:54.397278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb047dbd-481d-40f5-a3a7-8fac76fd73c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:54.397874 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047dbd-481d-40f5-a3a7-8fac76fd73c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:54.398151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047dbd-481d-40f5-a3a7-8fac76fd73c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:54.398454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb047dbd-481d-40f5-a3a7-8fac76fd73c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:54.460505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb047dbd-481d-40f5-a3a7-8fac76fd73c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:54.461962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb047dbd-481d-40f5-a3a7-8fac76fd73c0 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:54.471470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1344/2700] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:44:59.212144 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:59.274083 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e3bcd-0b47-41b6-b029-2e68fef1ab19 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:59.278838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:59.279451 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:59.279729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:59.280000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] PATH_INFO: `/v3/users/92db207d02614e1f820d812980366de1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:59.281127 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=92db207d02614e1f820d812980366de1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:59.283076 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:59.467674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92db207d02614e1f820d812980366de1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:59.539386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92db207d02614e1f820d812980366de1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:59.553099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92db207d02614e1f820d812980366de1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:59.557117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92db207d02614e1f820d812980366de1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:59.557117 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92db207d02614e1f820d812980366de1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:44:59.566973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1357/2701] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:44:59 2022] DELETE /identity/v3/users/92db207d02614e1f820d812980366de1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:44:59.571431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:59.571917 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:59.572045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:59.590994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:44:59.592191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:59.648950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1345/2702] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:44:59.658669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:44:59.660700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:44:59.717772 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadf0eb-9226-4c2f-a211-a6ddbba57a4d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:44:59.721453 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:44:59.722586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:44:59.722868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:44:59.723140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:44:59.732102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:44:59.733527 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:44:59.743580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1358/2703] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:45:00.157737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:00.159857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:00.172320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6c40-16a0-4206-99c2-6a3a5674357c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:00.178165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:00.178844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:00.179117 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:00.179384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] PATH_INFO: `/v3/projects/82c0466829d94e0d8bf039ad0a8df2c2` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:00.180540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c0466829d94e0d8bf039ad0a8df2c2)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:00.182471 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:00.282223 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c0466829d94e0d8bf039ad0a8df2c2', 'request_id': 'req-7988c523-70fc-41a6-85c2-fd5ae7e6178c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.289280 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c0466829d94e0d8bf039ad0a8df2c2', 'request_id': 'req-7988c523-70fc-41a6-85c2-fd5ae7e6178c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.367780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c0466829d94e0d8bf039ad0a8df2c2', 'request_id': 'req-7988c523-70fc-41a6-85c2-fd5ae7e6178c'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.379441 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] The token cache is being invalidate because project 82c0466829d94e0d8bf039ad0a8df2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:45:00.379733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.380367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.381828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1346/2704] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:45:00 2022] DELETE /identity/v3/projects/82c0466829d94e0d8bf039ad0a8df2c2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:00.385780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:00.395449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79b195-5484-4de2-8b2f-7cf61aaf11c2 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:00.404185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:00.451944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:00.452559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:00.452837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:00.453106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] PATH_INFO: `/v3/users/cc6239b2d2c74f83bfe7f45adc7e5e48` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:00.454215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6239b2d2c74f83bfe7f45adc7e5e48)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:00.456248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:00.774098 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6239b2d2c74f83bfe7f45adc7e5e48'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.819259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6239b2d2c74f83bfe7f45adc7e5e48'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.832893 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6239b2d2c74f83bfe7f45adc7e5e48'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.833539 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6239b2d2c74f83bfe7f45adc7e5e48'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.834148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8d797646-4d62-4faf-a32b-aa25321a6bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6239b2d2c74f83bfe7f45adc7e5e48'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:00.847712 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1359/2705] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:45:00 2022] DELETE /identity/v3/users/cc6239b2d2c74f83bfe7f45adc7e5e48 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:01.301371 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:01.302909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:01.318871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988c523-70fc-41a6-85c2-fd5ae7e6178c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:01.338971 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:01.379530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:01.380436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:01.381078 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:01.381374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] PATH_INFO: `/v3/projects/3d6465b0a8c54125aecaed0fe16287cf` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:01.382628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6465b0a8c54125aecaed0fe16287cf)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:01.384623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:01.482860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6465b0a8c54125aecaed0fe16287cf', 'request_id': 'req-de926de9-213b-4383-8f2c-9e235faa79b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:01.490472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6465b0a8c54125aecaed0fe16287cf', 'request_id': 'req-de926de9-213b-4383-8f2c-9e235faa79b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:01.574208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6465b0a8c54125aecaed0fe16287cf', 'request_id': 'req-de926de9-213b-4383-8f2c-9e235faa79b9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:01.586497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] The token cache is being invalidate because project 3d6465b0a8c54125aecaed0fe16287cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:45:01.586785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:01.587414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-de926de9-213b-4383-8f2c-9e235faa79b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:01.589042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1347/2706] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:45:01 2022] DELETE /identity/v3/projects/3d6465b0a8c54125aecaed0fe16287cf => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:01.595059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6451a-9edc-4b46-926e-b63f67c2f13f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:01.595530 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6451a-9edc-4b46-926e-b63f67c2f13f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:01.595630 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6451a-9edc-4b46-926e-b63f67c2f13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:01.617780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c5b6451a-9edc-4b46-926e-b63f67c2f13f None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:01.619012 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c5b6451a-9edc-4b46-926e-b63f67c2f13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:01.677062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1360/2707] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:01.686098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c6fb2-2899-4fba-b420-4e1ecf0d0168 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:01.686411 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c6fb2-2899-4fba-b420-4e1ecf0d0168 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:01.686691 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c6fb2-2899-4fba-b420-4e1ecf0d0168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:01.706148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a84c6fb2-2899-4fba-b420-4e1ecf0d0168 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:01.707377 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a84c6fb2-2899-4fba-b420-4e1ecf0d0168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:01.715227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1348/2708] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:01.719418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6451a-9edc-4b46-926e-b63f67c2f13f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:01.729567 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6451a-9edc-4b46-926e-b63f67c2f13f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:01.734130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:01.734755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:01.735027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:01.735290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:01.735919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:45:01.736211 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:01.738432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:01.749549 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1361/2709] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:45:01.753956 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c6fb2-2899-4fba-b420-4e1ecf0d0168 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:01.763091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c6fb2-2899-4fba-b420-4e1ecf0d0168 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:01.767678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:01.768257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:01.768564 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:01.768989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:01.769604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:01.771522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:01.849594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1349/2710] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:45:01 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:45:01.853742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:01.864512 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460b794-c190-43be-84ce-212839c09b38 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:01.881199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:01.920837 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:01.921406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:01.921959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:01.922554 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:01.923622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:01.926293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.013520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1362/2711] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:45:01 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:45:02.017683 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.028772 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4dc257-ec66-4f61-ac22-574b4bdce184 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.033362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.034011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.035083 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.035512 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.036146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.037711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.047470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1350/2712] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:45:02.051463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.061791 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404ca17-4271-48db-83fc-870bc674491a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.066601 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd177f92-8b84-4a60-838a-4f9a23a84abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.067189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177f92-8b84-4a60-838a-4f9a23a84abb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.067465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177f92-8b84-4a60-838a-4f9a23a84abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.067735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177f92-8b84-4a60-838a-4f9a23a84abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.068395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd177f92-8b84-4a60-838a-4f9a23a84abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.069962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd177f92-8b84-4a60-838a-4f9a23a84abb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.080563 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1363/2713] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:45:02.085139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.095031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557e38f-2e2d-4f73-a179-9821a7daddd7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.100323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.100954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.101231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.101498 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] PATH_INFO: `/v3/projects/cf5105d67ea74d9dae98b6f79e124b54/users/6dbb956b1e044040a397d7496ab2abfd/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.115556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5105d67ea74d9dae98b6f79e124b54, user_id=6dbb956b1e044040a397d7496ab2abfd, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.118172 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.185360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1351/2714] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:45:02 2022] PUT /identity/v3/projects/cf5105d67ea74d9dae98b6f79e124b54/users/6dbb956b1e044040a397d7496ab2abfd/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:02.203834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd0630-6102-46aa-ae25-cf0516e0fd04 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.204141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd0630-6102-46aa-ae25-cf0516e0fd04 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.204408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd0630-6102-46aa-ae25-cf0516e0fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.232786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0bcd0630-6102-46aa-ae25-cf0516e0fd04 None None] MFA Rules not processed for user `6dbb956b1e044040a397d7496ab2abfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:02.233991 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0bcd0630-6102-46aa-ae25-cf0516e0fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:02.297174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1364/2715] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:02.301964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.351768 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:02.365537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0e39f-7e9a-4a39-aeb2-402116da05b2 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.370470 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.371070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.371344 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.371611 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.372314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.374381 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.518370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1352/2716] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:45:02 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:45:02.523141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd0630-6102-46aa-ae25-cf0516e0fd04 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.533894 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd0630-6102-46aa-ae25-cf0516e0fd04 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.592160 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.592917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.593283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.593550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.594182 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.596189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.722377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1365/2717] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:45:02 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:45:02.726337 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.736639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e68f76-a6d7-4042-baac-09c85a5f670b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.741831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.742453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.742732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.743001 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.743630 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.745214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.756430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1353/2718] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:45:02.760742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.771520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e9172-979c-4460-9141-e6babff9490f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.776303 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.778871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.778871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.778871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] PATH_INFO: `/v3/projects/77d1ca24f041431a97ce2d0f2ec64ab0/users/6fd205ed66154ea6b61fa1e3e1e060bb/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.793274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d1ca24f041431a97ce2d0f2ec64ab0, user_id=6fd205ed66154ea6b61fa1e3e1e060bb, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.796573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.798629 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.800588 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:02.800884 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:02.874262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1366/2719] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:45:02 2022] PUT /identity/v3/projects/77d1ca24f041431a97ce2d0f2ec64ab0/users/6fd205ed66154ea6b61fa1e3e1e060bb/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:45:02.874860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e83e8-4c93-4f09-9e80-b72bbb71d5c8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.875358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.952349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b9b3-1b7f-49e4-b276-698871a99d51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.952955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b9b3-1b7f-49e4-b276-698871a99d51 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.953234 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b9b3-1b7f-49e4-b276-698871a99d51 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.953501 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b9b3-1b7f-49e4-b276-698871a99d51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.955293 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-d1c6b9b3-1b7f-49e4-b276-698871a99d51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:02.955640 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366beaa6-c18c-4cfa-a160-c415bb94b936 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.960514 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.961102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.961380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.961654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.962290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.963888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.967059 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6b9b3-1b7f-49e4-b276-698871a99d51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.968623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6b9b3-1b7f-49e4-b276-698871a99d51 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:02.973967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1367/2720] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:45:02.978205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:02.979650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1354/2721] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:45:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:45:02.988970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bb9a0-ecdd-4d9b-ae80-488fb78c2cd1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:02.993613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:02.994260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:02.994568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:02.994844 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] PATH_INFO: `/v3/projects/77d1ca24f041431a97ce2d0f2ec64ab0/users/6fd205ed66154ea6b61fa1e3e1e060bb/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:02.996797 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d1ca24f041431a97ce2d0f2ec64ab0, user_id=6fd205ed66154ea6b61fa1e3e1e060bb, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:02.999465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:03.010740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1368/2722] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:45:02 2022] PUT /identity/v3/projects/77d1ca24f041431a97ce2d0f2ec64ab0/users/6fd205ed66154ea6b61fa1e3e1e060bb/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:45:03.021913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e6af-6e32-45cb-bdd2-db0f7f146af3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:03.022089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e6af-6e32-45cb-bdd2-db0f7f146af3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:03.025617 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e6af-6e32-45cb-bdd2-db0f7f146af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:03.052023 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a0c5e6af-6e32-45cb-bdd2-db0f7f146af3 None None] MFA Rules not processed for user `6fd205ed66154ea6b61fa1e3e1e060bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:03.053275 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a0c5e6af-6e32-45cb-bdd2-db0f7f146af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:03.087600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:03.122012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1355/2723] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:03.134580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:03.136290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:03.201330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ed5b7-3cfd-44f1-9b09-9b242e84a3cd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:03.205014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:03.205614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:03.205943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:03.206371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:03.215919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:03.217366 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:03.227850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1369/2724] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:45:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:45:03.361328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5e6af-6e32-45cb-bdd2-db0f7f146af3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:03.372542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5e6af-6e32-45cb-bdd2-db0f7f146af3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:03.376250 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52d5ed-cf2a-44c6-91fb-3f362c1bf002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:03.376911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d5ed-cf2a-44c6-91fb-3f362c1bf002 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:03.377247 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d5ed-cf2a-44c6-91fb-3f362c1bf002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:03.377576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d5ed-cf2a-44c6-91fb-3f362c1bf002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:03.436466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52d5ed-cf2a-44c6-91fb-3f362c1bf002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:03.437924 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52d5ed-cf2a-44c6-91fb-3f362c1bf002 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:03.447228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1356/2725] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:45:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:45:03.955795 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:04.021206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e7e5c-109e-4c04-a071-b49b8355e0ca service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:04.025884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:04.026540 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:04.026843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:04.027121 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] PATH_INFO: `/v3/users/6dbb956b1e044040a397d7496ab2abfd` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:04.028253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dbb956b1e044040a397d7496ab2abfd)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:04.030177 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:04.053838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=2,keystone:SELECT=95,keystone:DELETE=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:04.232314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dbb956b1e044040a397d7496ab2abfd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:04.281705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dbb956b1e044040a397d7496ab2abfd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:04.294613 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dbb956b1e044040a397d7496ab2abfd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:04.295304 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dbb956b1e044040a397d7496ab2abfd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:04.295878 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dbb956b1e044040a397d7496ab2abfd'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:04.308171 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1370/2726] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:45:03 2022] DELETE /identity/v3/users/6dbb956b1e044040a397d7496ab2abfd => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:04.312624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:04.312907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:04.313182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:04.332769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:04.333974 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:04.390237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1357/2727] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:04.400360 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:04.401941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:04.459973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b19db-4c24-4922-95d4-2929dbd8c612 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:04.463576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:04.464196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:04.464494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:04.464760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:04.474154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:04.475637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:04.484914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1371/2728] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:45:04.981025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:04.982567 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:04.999966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7130d4-6dd6-4bd1-acfd-6df5e27c79e4 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:05.006025 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:05.007751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:05.008106 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:05.008284 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] PATH_INFO: `/v3/projects/cf5105d67ea74d9dae98b6f79e124b54` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:05.009427 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5105d67ea74d9dae98b6f79e124b54)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:05.011426 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:05.117652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5105d67ea74d9dae98b6f79e124b54', 'request_id': 'req-a81f4b57-b683-45ac-9449-48ff4ddd494e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.124346 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5105d67ea74d9dae98b6f79e124b54', 'request_id': 'req-a81f4b57-b683-45ac-9449-48ff4ddd494e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.213600 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5105d67ea74d9dae98b6f79e124b54', 'request_id': 'req-a81f4b57-b683-45ac-9449-48ff4ddd494e'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.218273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:05.226185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] The token cache is being invalidate because project cf5105d67ea74d9dae98b6f79e124b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:45:05.226465 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.227116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.228594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1358/2729] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:45:04 2022] DELETE /identity/v3/projects/cf5105d67ea74d9dae98b6f79e124b54 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:05.232638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:05.243043 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a8207-8828-412d-848a-d62bd20cfec8 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:05.298495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:05.300109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:05.300109 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:05.300354 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] PATH_INFO: `/v3/users/6fd205ed66154ea6b61fa1e3e1e060bb` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:05.301459 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd205ed66154ea6b61fa1e3e1e060bb)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:05.303397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:05.431011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:05.433673 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd205ed66154ea6b61fa1e3e1e060bb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.488088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd205ed66154ea6b61fa1e3e1e060bb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.500014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd205ed66154ea6b61fa1e3e1e060bb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.500665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd205ed66154ea6b61fa1e3e1e060bb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.501442 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0df9ff3c-b977-468a-a4eb-72c395316fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd205ed66154ea6b61fa1e3e1e060bb'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:05.513637 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1372/2730] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:45:05 2022] DELETE /identity/v3/users/6fd205ed66154ea6b61fa1e3e1e060bb => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:05.925610 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:05.927349 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:05.940876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f4b57-b683-45ac-9449-48ff4ddd494e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:06.004764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:06.005389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.005667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.006946 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] PATH_INFO: `/v3/projects/77d1ca24f041431a97ce2d0f2ec64ab0` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.007641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d1ca24f041431a97ce2d0f2ec64ab0)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:06.010210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:06.101615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d1ca24f041431a97ce2d0f2ec64ab0', 'request_id': 'req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:06.112251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d1ca24f041431a97ce2d0f2ec64ab0', 'request_id': 'req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:06.190808 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d1ca24f041431a97ce2d0f2ec64ab0', 'request_id': 'req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:06.202502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] The token cache is being invalidate because project 77d1ca24f041431a97ce2d0f2ec64ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:45:06.202799 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:06.203447 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-1e7a08ca-d4c3-4a98-9c34-983f3aab87d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:45:06.204923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1359/2731] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:45:05 2022] DELETE /identity/v3/projects/77d1ca24f041431a97ce2d0f2ec64ab0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:06.210564 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd33437-febc-435e-84a7-75dabb7fb0a4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.210943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd33437-febc-435e-84a7-75dabb7fb0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.211282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd33437-febc-435e-84a7-75dabb7fb0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.232802 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dcd33437-febc-435e-84a7-75dabb7fb0a4 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:06.233924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dcd33437-febc-435e-84a7-75dabb7fb0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:06.296782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1373/2732] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:06.361065 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce143ce-4a40-407f-9361-a4a7d12954c7 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.361405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce143ce-4a40-407f-9361-a4a7d12954c7 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.361593 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce143ce-4a40-407f-9361-a4a7d12954c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.381021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-9ce143ce-4a40-407f-9361-a4a7d12954c7 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:06.382481 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-9ce143ce-4a40-407f-9361-a4a7d12954c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:06.394546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1360/2733] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:06.396111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd33437-febc-435e-84a7-75dabb7fb0a4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:06.405745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd33437-febc-435e-84a7-75dabb7fb0a4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:06.411298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:06.412088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.412368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.412685 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.413359 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:45:06.413663 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:06.415344 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:06.427439 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1374/2734] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:45:06.431888 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce143ce-4a40-407f-9361-a4a7d12954c7 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:06.441548 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce143ce-4a40-407f-9361-a4a7d12954c7 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:06.446285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:06.446904 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.447180 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.447450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.448075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:06.450208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:06.515835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1361/2735] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:45:06 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:45:06.517272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:06.528494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a413c1-12ad-459a-9d81-65e597a590e7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:06.588399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:06.588998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.589275 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.589545 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.590191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:06.592128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:06.674274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1375/2736] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:45:06 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:45:06.675837 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:06.686102 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fb690-4a10-43f2-91c2-fc504e687844 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:06.691273 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:06.692120 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.692386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.692648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.693274 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:06.694887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:06.705021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1362/2737] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:45:06.709906 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:06.720199 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071051a-43ef-4583-a5fb-ebb8fffa3666 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:06.725081 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16300840-70df-4d65-970a-e64b3fafb94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:06.725674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300840-70df-4d65-970a-e64b3fafb94a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.725951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300840-70df-4d65-970a-e64b3fafb94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.726222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300840-70df-4d65-970a-e64b3fafb94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.726957 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16300840-70df-4d65-970a-e64b3fafb94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:06.728533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16300840-70df-4d65-970a-e64b3fafb94a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:06.739896 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1376/2738] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:45:06.744075 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:06.755338 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1c9a1-3ed7-4234-91fc-b362aeb484b5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:06.760268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:06.760967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.761248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.761528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] PATH_INFO: `/v3/projects/c3703fa0da4a45d8b49bf637d12e3f21/users/44565ea5afce40aba02902b89b76a5f1/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.777958 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3703fa0da4a45d8b49bf637d12e3f21, user_id=44565ea5afce40aba02902b89b76a5f1, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:06.780682 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:06.838034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1363/2739] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:45:06 2022] PUT /identity/v3/projects/c3703fa0da4a45d8b49bf637d12e3f21/users/44565ea5afce40aba02902b89b76a5f1/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:45:06.843239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46477a10-0c3f-4f4b-b281-fb9e18130479 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:06.843556 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46477a10-0c3f-4f4b-b281-fb9e18130479 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:06.843832 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46477a10-0c3f-4f4b-b281-fb9e18130479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:06.864660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-46477a10-0c3f-4f4b-b281-fb9e18130479 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:06.865902 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-46477a10-0c3f-4f4b-b281-fb9e18130479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:06.887996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:06.934971 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1377/2740] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:06.942370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:06.944387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:06.967521 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:07.008627 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380f0fb-7148-4936-8990-2fe9f446b3f8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:07.012375 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:07.012969 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:07.013246 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:07.013510 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:07.023359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:07.024807 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:07.034819 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1364/2741] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:45:08.387477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac579aa-0fc8-4863-b354-15bc34ca16c8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:08.387477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac579aa-0fc8-4863-b354-15bc34ca16c8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:08.388107 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac579aa-0fc8-4863-b354-15bc34ca16c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:08.430576 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-eac579aa-0fc8-4863-b354-15bc34ca16c8 None None] MFA Rules not processed for user `44565ea5afce40aba02902b89b76a5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:08.433339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eac579aa-0fc8-4863-b354-15bc34ca16c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:08.507887 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1378/2742] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:08.518674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:08.520330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:08.581793 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa1f85-300b-4be3-b3be-8b5de24c4dca service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:08.585532 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258e70a-0dec-4b67-a140-a7f3a9f03537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:08.586123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258e70a-0dec-4b67-a140-a7f3a9f03537 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:08.586452 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258e70a-0dec-4b67-a140-a7f3a9f03537 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:08.586712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258e70a-0dec-4b67-a140-a7f3a9f03537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:08.595215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258e70a-0dec-4b67-a140-a7f3a9f03537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:08.596737 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258e70a-0dec-4b67-a140-a7f3a9f03537 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:08.605867 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1365/2743] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:45:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:45:10.232815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:10.233358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:10.233358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:10.253612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] MFA Rules not processed for user `44565ea5afce40aba02902b89b76a5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:10.254850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:10.261708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1379/2744] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:18.601938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=43 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:20.246709 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=50 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:32.633071 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47b397-7535-4373-b33d-e8bfa1428c57 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:32.633544 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47b397-7535-4373-b33d-e8bfa1428c57 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:32.633652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47b397-7535-4373-b33d-e8bfa1428c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:32.652689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-bd47b397-7535-4373-b33d-e8bfa1428c57 None None] MFA Rules not processed for user `44565ea5afce40aba02902b89b76a5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:32.653908 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bd47b397-7535-4373-b33d-e8bfa1428c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:32.660642 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1366/2745] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:45:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:42.647162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:45:50.820834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:50.822223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:50.892811 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d16b3c-fd20-4862-8084-4ffbdd12efe7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:50.898122 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:50.898748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:50.899024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:50.899289 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:50.901016 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:50.912316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:50.913868 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:50.924777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1380/2746] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:45:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:45:58.214777 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df4b84-1b46-4b53-b696-7eeebd7c3e2a None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:58.215285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df4b84-1b46-4b53-b696-7eeebd7c3e2a None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:58.215405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df4b84-1b46-4b53-b696-7eeebd7c3e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:58.234694 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-15df4b84-1b46-4b53-b696-7eeebd7c3e2a None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:45:58.235882 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-15df4b84-1b46-4b53-b696-7eeebd7c3e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:58.291173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1367/2747] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:45:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:45:58.439029 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:45:58.440504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:45:58.510266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e343867-3f9b-4ad0-a207-555c98ac5965 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:45:58.515243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:45:58.515831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:45:58.516108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:45:58.516377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:45:58.525114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:45:58.526717 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:45:58.534607 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1381/2748] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:45:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:46:08.284019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=25 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:08.531456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=64 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:15.724754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6978a-9128-425d-9e96-7561044d2692 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:15.725227 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6978a-9128-425d-9e96-7561044d2692 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:15.725322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6978a-9128-425d-9e96-7561044d2692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:15.744598 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-05a6978a-9128-425d-9e96-7561044d2692 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:46:15.745779 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-05a6978a-9128-425d-9e96-7561044d2692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:15.752831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1368/2749] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 02:46:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:46:15.765243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:15.775847 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11241a04-0596-4a53-9726-75a17aac25d2 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:15.780828 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:15.781407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:15.781682 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:15.781952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:15.790204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:15.791785 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:15.799664 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1382/2750] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:46:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:46:18.664005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6978a-9128-425d-9e96-7561044d2692 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:18.676723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6978a-9128-425d-9e96-7561044d2692 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:18.682283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:18.682641 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:18.682915 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:18.683185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] PATH_INFO: `/v3/users/44565ea5afce40aba02902b89b76a5f1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:18.684340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44565ea5afce40aba02902b89b76a5f1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:18.686418 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:18.859886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44565ea5afce40aba02902b89b76a5f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:18.932121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44565ea5afce40aba02902b89b76a5f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:18.944374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44565ea5afce40aba02902b89b76a5f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:18.952087 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44565ea5afce40aba02902b89b76a5f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:18.952718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-54ce7633-632c-4823-8fb6-bbb8a5d2cb71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44565ea5afce40aba02902b89b76a5f1'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:18.958625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1369/2751] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:46:18 2022] DELETE /identity/v3/users/44565ea5afce40aba02902b89b76a5f1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:46:19.207820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:19.219522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e009-61a4-4ab8-b4eb-254a87a6a209 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:19.279816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:19.280455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:19.280730 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:19.280996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] PATH_INFO: `/v3/projects/c3703fa0da4a45d8b49bf637d12e3f21` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:19.282111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3703fa0da4a45d8b49bf637d12e3f21)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:19.284057 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:19.378193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3703fa0da4a45d8b49bf637d12e3f21', 'request_id': 'req-4b4c654e-ed3e-458a-b5c7-0b822d19af47'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:19.429295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3703fa0da4a45d8b49bf637d12e3f21', 'request_id': 'req-4b4c654e-ed3e-458a-b5c7-0b822d19af47'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:19.435705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3703fa0da4a45d8b49bf637d12e3f21', 'request_id': 'req-4b4c654e-ed3e-458a-b5c7-0b822d19af47'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:19.445903 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] The token cache is being invalidate because project c3703fa0da4a45d8b49bf637d12e3f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:46:19.446191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:19.447982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:46:19.449447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1383/2752] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:46:19 2022] DELETE /identity/v3/projects/c3703fa0da4a45d8b49bf637d12e3f21 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:46:28.955131 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:29.442969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:46.460088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:46.460612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:46.460714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:46.479511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] MFA Rules not processed for user `f3a60f8c44524cc4a7a394060e619b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:46:46.480714 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:46.535407 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1370/2753] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:46:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:46:46.848722 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:46:46.850125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:46.906414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c654e-ed3e-458a-b5c7-0b822d19af47 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:46:46.909974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:46:46.910580 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:46:46.910854 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:46:46.911119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:46:46.912827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:46:46.924436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:46:46.925889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:46:46.935128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1384/2754] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:46:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:46:56.528748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=25 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:46:56.930421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=26 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:47:01.508328 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:47:01.509864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:01.583752 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3fedde-d874-4169-a86e-18d87e803985 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:47:01.589885 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:47:01.590542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:47:01.590820 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:47:01.591090 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:47:01.592860 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:47:01.606389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:47:01.607994 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:47:01.620094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1371/2755] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:47:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:47:11.614565 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=34 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:48:16.670595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:48:16.672998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:48:16.731929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5de08-9237-4b1d-80d9-891c73dd91c5 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:48:16.735568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a737ae-f79b-4cfc-9505-420c6db1ee72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:48:16.736168 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a737ae-f79b-4cfc-9505-420c6db1ee72 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:48:16.736447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a737ae-f79b-4cfc-9505-420c6db1ee72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:48:16.736718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a737ae-f79b-4cfc-9505-420c6db1ee72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:48:16.738491 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c0a737ae-f79b-4cfc-9505-420c6db1ee72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:48:16.752774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a737ae-f79b-4cfc-9505-420c6db1ee72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:48:16.754217 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a737ae-f79b-4cfc-9505-420c6db1ee72 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:48:16.765504 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1385/2756] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:48:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:48:18.433479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:48:18.434898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:48:18.446672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb121cd-7ba1-469d-98a9-94323ab33d7a service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:48:18.451257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:48:18.451901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:48:18.452181 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:48:18.452456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:48:18.454191 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:48:18.467632 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:48:18.469122 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:48:18.480091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1372/2757] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:48:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:48:26.761861 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=26 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:48:28.475200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:48:31.411266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:48:31.411829 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:48:31.411944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:48:31.432994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] MFA Rules not processed for user `f3a60f8c44524cc4a7a394060e619b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:48:31.437738 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:48:31.441378 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1386/2758] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:48:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:48:35.098219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:48:35.107936 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0394e-f0a7-4574-a3a3-4ad8484c3766 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:48:35.112719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:48:35.112719 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:48:35.113595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:48:35.113595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:48:35.115291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:48:35.126432 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:48:35.127830 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:48:35.135954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1373/2759] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:48:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:48:41.426787 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:48:45.132262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:49:51.355875 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:49:51.357299 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:51.373724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b7b8a-03df-448d-8f4d-e58a2e820a82 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:49:51.379294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dd168-20c1-49ec-985f-50513f656693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:49:51.379883 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dd168-20c1-49ec-985f-50513f656693 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:49:51.380164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dd168-20c1-49ec-985f-50513f656693 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:49:51.380435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dd168-20c1-49ec-985f-50513f656693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:49:51.382124 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-240dd168-20c1-49ec-985f-50513f656693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:49:51.393428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dd168-20c1-49ec-985f-50513f656693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:49:51.411225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dd168-20c1-49ec-985f-50513f656693 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:49:51.411225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1387/2760] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:49:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:50:01.399860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:50:06.049255 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:50:06.063085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccab8c7-69fd-4e64-849a-6735f6214dc0 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:50:06.072310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:50:06.073396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:50:06.073798 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:50:06.074066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:50:06.075876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:50:06.089242 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:50:06.091228 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:50:06.107036 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1374/2761] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:50:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:50:11.237314 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532c6c-7215-409d-80f8-f9e51c7844e9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:50:11.238923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532c6c-7215-409d-80f8-f9e51c7844e9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:50:11.238923 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532c6c-7215-409d-80f8-f9e51c7844e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:50:11.256833 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-35532c6c-7215-409d-80f8-f9e51c7844e9 None None] MFA Rules not processed for user `f3a60f8c44524cc4a7a394060e619b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:50:11.258018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-35532c6c-7215-409d-80f8-f9e51c7844e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:50:11.264973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1388/2762] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:50:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:50:11.947991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:50:11.949738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:50:12.019457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23935c-3759-4a65-8892-083a37e909da service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:50:12.024491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:50:12.025161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:50:12.025367 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:50:12.025637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:50:12.027603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:50:12.038835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:50:12.040391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:50:12.050725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1375/2763] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:50:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:50:21.252071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:50:22.046543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=38 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:50:22.515219 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35532c6c-7215-409d-80f8-f9e51c7844e9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:50:22.536273 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35532c6c-7215-409d-80f8-f9e51c7844e9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:50:22.541700 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:50:22.542290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:50:22.542747 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:50:22.543274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:50:22.544843 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:50:22.567884 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:50:22.569481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:50:22.581148 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1389/2764] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:50:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:50:32.576285 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=13 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:51:38.658476 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:38.659862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:38.743907 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064b9bf-24f5-4096-8796-42e94eefa32d service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:38.749208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:38.749804 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:38.750081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:38.750981 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:38.752725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:38.764040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:38.765657 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:38.775891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1376/2765] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:51:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:51:48.771645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=37 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:51:53.562282 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:53.584403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c01bc-c659-4e49-8161-3dd431d315b5 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:53.587962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:53.588550 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:53.588860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:53.589130 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:53.590867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:53.674559 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:53.675970 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:53.694867 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1390/2766] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:51:57.187414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:57.198721 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef790b29-1362-4610-9c09-4132f85d8a08 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:57.203898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:57.205121 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:57.205607 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:57.205939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:57.207764 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:57.219290 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:57.221596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:57.231473 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1377/2767] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:51:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:51:57.319143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:57.320678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:57.334834 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b660c-74a1-4275-8704-16f6510df042 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:57.339996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:57.340620 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:57.340897 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:57.341196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] PATH_INFO: `/v3/users/f3a60f8c44524cc4a7a394060e619b9d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:57.342295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a60f8c44524cc4a7a394060e619b9d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:57.344278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:57.487773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a60f8c44524cc4a7a394060e619b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:57.542396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a60f8c44524cc4a7a394060e619b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:57.555193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a60f8c44524cc4a7a394060e619b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:57.555826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a60f8c44524cc4a7a394060e619b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:57.556421 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e9fa7cc2-28e0-4b08-a806-bc599edd6502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a60f8c44524cc4a7a394060e619b9d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:57.569589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1391/2768] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:51:57 2022] DELETE /identity/v3/users/f3a60f8c44524cc4a7a394060e619b9d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:51:57.844667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:57.858182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aacc40-170f-422e-be55-9559e3014227 service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:57.915092 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:57.915716 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:57.915993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:57.916264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] PATH_INFO: `/v3/projects/570f253b2653418dabc6566536e14353` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:57.917387 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=570f253b2653418dabc6566536e14353)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:57.919339 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:58.022379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570f253b2653418dabc6566536e14353', 'request_id': 'req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:58.028604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570f253b2653418dabc6566536e14353', 'request_id': 'req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:58.125188 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570f253b2653418dabc6566536e14353', 'request_id': 'req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:58.135999 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] The token cache is being invalidate because project 570f253b2653418dabc6566536e14353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:51:58.136286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:58.136909 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ab5e79bc-dd2c-4278-a3e0-62c87479d2ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:51:58.138397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1378/2769] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:51:57 2022] DELETE /identity/v3/projects/570f253b2653418dabc6566536e14353 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:51:58.143326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805e970-739c-4a15-8cdb-55fe4f2ebd6a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.147097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805e970-739c-4a15-8cdb-55fe4f2ebd6a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.147097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805e970-739c-4a15-8cdb-55fe4f2ebd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.177411 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:51:58.201703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2805e970-739c-4a15-8cdb-55fe4f2ebd6a None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:51:58.202886 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2805e970-739c-4a15-8cdb-55fe4f2ebd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:58.261617 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1392/2770] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:51:58.269649 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eda8f-8bb2-465b-86de-c8dfaf4623fc None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.270123 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eda8f-8bb2-465b-86de-c8dfaf4623fc None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.270244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eda8f-8bb2-465b-86de-c8dfaf4623fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.290264 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-e68eda8f-8bb2-465b-86de-c8dfaf4623fc None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:51:58.291482 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-e68eda8f-8bb2-465b-86de-c8dfaf4623fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:58.300162 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1379/2771] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:51:58.304517 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805e970-739c-4a15-8cdb-55fe4f2ebd6a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:58.314243 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805e970-739c-4a15-8cdb-55fe4f2ebd6a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:58.318759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:58.319340 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.319891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.319891 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.320481 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:51:58.320776 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:58.322334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:58.333507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1393/2772] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:51:58.337646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68eda8f-8bb2-465b-86de-c8dfaf4623fc None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:58.347653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68eda8f-8bb2-465b-86de-c8dfaf4623fc None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:58.352434 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:58.353031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.353291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.353562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.354193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:58.356107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:58.469586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1380/2773] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:51:58 2022] POST /identity/v3/projects => generated 405 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:51:58.474337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:58.484337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d3271-7d07-4edc-bc59-bfa99902921b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:58.539781 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:58.540361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.540634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.540899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.541509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:58.543432 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:58.673857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1394/2774] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:51:58 2022] POST /identity/v3/users => generated 303 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:51:58.677976 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:58.688189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b3e23-f7bd-4b0f-89cc-b612641e5deb admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:58.693019 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:58.693624 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.693876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.694139 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.694801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:58.696369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:58.707285 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1381/2775] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:51:58.711395 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:58.722345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abec3b-ba3a-4711-b490-b45a3b22f661 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:58.727131 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364df45-85cb-43cc-81b0-35b345aac3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:58.727721 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364df45-85cb-43cc-81b0-35b345aac3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.728002 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364df45-85cb-43cc-81b0-35b345aac3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.728274 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364df45-85cb-43cc-81b0-35b345aac3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.728911 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364df45-85cb-43cc-81b0-35b345aac3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:58.730584 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364df45-85cb-43cc-81b0-35b345aac3f6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:58.741232 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1395/2776] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:51:58.745314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:58.756185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a366e-606c-41e0-80f8-2ff0f3c1b001 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:58.760759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:58.761384 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.761653 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.761919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] PATH_INFO: `/v3/projects/e958af440c0843bc95ddc2cd126155a5/users/793e583eb32842a4970257cd40a8abd4/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.777128 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e958af440c0843bc95ddc2cd126155a5, user_id=793e583eb32842a4970257cd40a8abd4, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:58.779923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:58.862215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1382/2777] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:51:58 2022] PUT /identity/v3/projects/e958af440c0843bc95ddc2cd126155a5/users/793e583eb32842a4970257cd40a8abd4/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:51:58.867694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04bc58-10f5-4a1e-9ca4-b414c1963f14 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:58.867973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04bc58-10f5-4a1e-9ca4-b414c1963f14 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:58.868242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04bc58-10f5-4a1e-9ca4-b414c1963f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:58.887618 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7e04bc58-10f5-4a1e-9ca4-b414c1963f14 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:51:58.888814 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7e04bc58-10f5-4a1e-9ca4-b414c1963f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:58.944815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:51:58.953206 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1396/2778] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:51:58.963366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:51:58.965097 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:51:58.983148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:51:59.034754 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c6bce-71fb-4615-a976-31e17765617c admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:51:59.038508 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:51:59.039103 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:51:59.039379 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:51:59.039675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:51:59.048614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:51:59.050208 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:51:59.059502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1383/2779] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:52:00.285055 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa6c83-230b-4658-990b-e79947f0dd42 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:00.285627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa6c83-230b-4658-990b-e79947f0dd42 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:00.285627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa6c83-230b-4658-990b-e79947f0dd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:00.319166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f1aa6c83-230b-4658-990b-e79947f0dd42 None None] MFA Rules not processed for user `793e583eb32842a4970257cd40a8abd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:00.320960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f1aa6c83-230b-4658-990b-e79947f0dd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:00.387704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1397/2780] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:00.392914 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:00.402542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cc3c8-1bca-4d0a-b64c-7e03497ebfc6 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:00.407147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:00.408835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:00.408835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:00.408835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:00.409143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:00.410948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:00.521995 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1384/2781] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:52:00 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:00.525934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa6c83-230b-4658-990b-e79947f0dd42 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:00.536973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa6c83-230b-4658-990b-e79947f0dd42 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:00.596462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:00.597061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:00.597336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:00.597678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:00.598292 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:00.600229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:00.679919 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1398/2782] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:52:00 2022] POST /identity/v3/users => generated 308 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:00.683916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:00.695672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6439b3-1be2-4174-a3f5-e2f1335d75c8 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:00.700210 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:00.700816 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:00.701091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:00.701357 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:00.701989 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:00.703749 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:00.714082 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1385/2783] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:00.718073 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:00.728804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bf4b4-bb18-43fd-9d50-1bfb5e033237 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:00.733450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:00.734082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:00.734377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:00.734665 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] PATH_INFO: `/v3/projects/58d4fa1e73964ae5a0361b582e8ffdff/users/9197f0f8cfe84a0bbcbd4236ac990cea/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:00.750027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d4fa1e73964ae5a0361b582e8ffdff, user_id=9197f0f8cfe84a0bbcbd4236ac990cea, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:00.753614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:00.822087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1399/2784] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:00 2022] PUT /identity/v3/projects/58d4fa1e73964ae5a0361b582e8ffdff/users/9197f0f8cfe84a0bbcbd4236ac990cea/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:00.826214 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:00.836604 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1192ab7-23f3-4b99-856c-de50cb943a01 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:00.891631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8ef03-f79a-4b7b-b70c-a0e3f4e72b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:00.892219 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8ef03-f79a-4b7b-b70c-a0e3f4e72b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:00.892497 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8ef03-f79a-4b7b-b70c-a0e3f4e72b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:00.892766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8ef03-f79a-4b7b-b70c-a0e3f4e72b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:00.893397 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8ef03-f79a-4b7b-b70c-a0e3f4e72b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:00.895011 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8ef03-f79a-4b7b-b70c-a0e3f4e72b8d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:00.904388 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1386/2785] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:00.908635 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:00.919015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25ff0c-e45f-47b5-b7d5-c9bc0153a604 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:00.924296 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:00.924948 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:00.925228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:00.925998 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] PATH_INFO: `/v3/projects/58d4fa1e73964ae5a0361b582e8ffdff/users/9197f0f8cfe84a0bbcbd4236ac990cea/roles/1b9fb565648c49388166fe130739891f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:00.927469 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d4fa1e73964ae5a0361b582e8ffdff, user_id=9197f0f8cfe84a0bbcbd4236ac990cea, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:00.930587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:00.942569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1400/2786] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:00 2022] PUT /identity/v3/projects/58d4fa1e73964ae5a0361b582e8ffdff/users/9197f0f8cfe84a0bbcbd4236ac990cea/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:02.175132 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f7af-eb3e-425a-b29d-a449f8a95a4f None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:02.175647 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f7af-eb3e-425a-b29d-a449f8a95a4f None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:02.175843 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f7af-eb3e-425a-b29d-a449f8a95a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:02.205039 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4b80f7af-eb3e-425a-b29d-a449f8a95a4f None None] MFA Rules not processed for user `9197f0f8cfe84a0bbcbd4236ac990cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:02.206475 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4b80f7af-eb3e-425a-b29d-a449f8a95a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:02.270478 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:02.280699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1387/2787] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:02 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:03.483532 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:03.488176 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:03.538087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:03.561474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e737a7-6cf6-4493-b9dd-ca6dc7660f37 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:03.567493 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:03.568085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:03.568363 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:03.568627 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:03.570346 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:03.582174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:03.583763 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:03.596179 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1401/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:52:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:52:03.597538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80f7af-eb3e-425a-b29d-a449f8a95a4f None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:03.663880 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80f7af-eb3e-425a-b29d-a449f8a95a4f None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:03.668393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:03.669046 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:03.669304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:03.669568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] PATH_INFO: `/v3/users/793e583eb32842a4970257cd40a8abd4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:03.670688 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793e583eb32842a4970257cd40a8abd4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:03.672585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:03.875757 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793e583eb32842a4970257cd40a8abd4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:03.965391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793e583eb32842a4970257cd40a8abd4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:03.978964 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793e583eb32842a4970257cd40a8abd4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:03.986479 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793e583eb32842a4970257cd40a8abd4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:03.986939 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793e583eb32842a4970257cd40a8abd4'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:03.992723 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1388/2789] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:52:03 2022] DELETE /identity/v3/users/793e583eb32842a4970257cd40a8abd4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:04.233307 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:04.245042 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e37f56-ae75-4d7a-9a83-cff9573e6a6b service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:04.304283 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:04.304934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:04.305213 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:04.305483 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] PATH_INFO: `/v3/projects/e958af440c0843bc95ddc2cd126155a5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:04.306662 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e958af440c0843bc95ddc2cd126155a5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:04.311159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:04.409670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e958af440c0843bc95ddc2cd126155a5', 'request_id': 'req-06c30dce-2675-4002-b072-db9af56977fc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.479520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e958af440c0843bc95ddc2cd126155a5', 'request_id': 'req-06c30dce-2675-4002-b072-db9af56977fc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.485783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e958af440c0843bc95ddc2cd126155a5', 'request_id': 'req-06c30dce-2675-4002-b072-db9af56977fc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.496128 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] The token cache is being invalidate because project e958af440c0843bc95ddc2cd126155a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:52:04.500334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.500334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.500334 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1402/2790] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:52:04 2022] DELETE /identity/v3/projects/e958af440c0843bc95ddc2cd126155a5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:04.502717 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:04.504133 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:04.517805 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1069-e00b-4bd4-94cc-dee084ee2f01 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:04.572802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:04.573416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:04.573689 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:04.573955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] PATH_INFO: `/v3/users/9197f0f8cfe84a0bbcbd4236ac990cea` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:04.575088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9197f0f8cfe84a0bbcbd4236ac990cea)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:04.577000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:04.590366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:04.828008 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9197f0f8cfe84a0bbcbd4236ac990cea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.896089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9197f0f8cfe84a0bbcbd4236ac990cea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.908400 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9197f0f8cfe84a0bbcbd4236ac990cea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.915442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9197f0f8cfe84a0bbcbd4236ac990cea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.916109 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ea658401-9468-4ebf-adc4-777ff9186975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9197f0f8cfe84a0bbcbd4236ac990cea'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:04.922226 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1389/2791] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:52:04 2022] DELETE /identity/v3/users/9197f0f8cfe84a0bbcbd4236ac990cea => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:52:05.176023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:05.177558 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:05.192225 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c30dce-2675-4002-b072-db9af56977fc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:05.249599 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:05.250222 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:05.250536 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:05.250838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] PATH_INFO: `/v3/projects/58d4fa1e73964ae5a0361b582e8ffdff` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:05.253248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d4fa1e73964ae5a0361b582e8ffdff)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:05.255337 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:05.257909 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:05.392414 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d4fa1e73964ae5a0361b582e8ffdff', 'request_id': 'req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:05.470480 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d4fa1e73964ae5a0361b582e8ffdff', 'request_id': 'req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:05.475342 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d4fa1e73964ae5a0361b582e8ffdff', 'request_id': 'req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:05.487053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] The token cache is being invalidate because project 58d4fa1e73964ae5a0361b582e8ffdff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:52:05.487639 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:05.487967 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1bffbb96-84b4-4a2e-b10e-58bb917f08b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:05.489463 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1403/2792] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:52:05 2022] DELETE /identity/v3/projects/58d4fa1e73964ae5a0361b582e8ffdff => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:05.494466 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6a744-4632-405d-8e0b-ce93c379c99c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:05.499101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6a744-4632-405d-8e0b-ce93c379c99c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:05.499101 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6a744-4632-405d-8e0b-ce93c379c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:05.514370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-eee6a744-4632-405d-8e0b-ce93c379c99c None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:05.515606 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-eee6a744-4632-405d-8e0b-ce93c379c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:05.573322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1390/2793] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:05.581154 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1eb1f-cdc8-4089-bb98-5223a1837430 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:05.581435 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1eb1f-cdc8-4089-bb98-5223a1837430 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:05.581706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1eb1f-cdc8-4089-bb98-5223a1837430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:05.601418 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4aa1eb1f-cdc8-4089-bb98-5223a1837430 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:05.602631 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4aa1eb1f-cdc8-4089-bb98-5223a1837430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:05.610269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1404/2794] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:05.614910 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6a744-4632-405d-8e0b-ce93c379c99c None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:05.624391 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6a744-4632-405d-8e0b-ce93c379c99c None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:05.629047 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:05.629631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:05.629916 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:05.630204 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:05.630856 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:52:05.631156 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:05.632732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:05.643899 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1391/2795] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:52:05.648035 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1eb1f-cdc8-4089-bb98-5223a1837430 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:05.657169 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1eb1f-cdc8-4089-bb98-5223a1837430 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:05.662436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:05.663011 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:05.663281 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:05.663544 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:05.664151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:05.666007 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:05.797960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1405/2796] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:52:05 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:05.803085 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:05.812962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522561b-e7b5-45c5-b8a9-e2daf0966eff admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:05.867828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:05.868460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:05.868674 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:05.868940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:05.869538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:05.871449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.042988 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1392/2797] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:52:05 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:06.047071 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.059137 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ebdac-16c6-4b70-ad2a-e4a0f9ff9b50 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.062474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.063062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.063651 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.064036 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.064960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.066562 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.076528 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1406/2798] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:06.081179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.091066 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739470-b8be-428c-aefa-9e2c4dd79ed4 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.096945 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a979114-a518-4db3-994e-e76f257dc83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.099389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979114-a518-4db3-994e-e76f257dc83b admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.099389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979114-a518-4db3-994e-e76f257dc83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.099389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979114-a518-4db3-994e-e76f257dc83b admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.099389 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a979114-a518-4db3-994e-e76f257dc83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.100310 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a979114-a518-4db3-994e-e76f257dc83b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.102493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:06.111931 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1393/2799] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:06.116052 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.126112 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3f3ab-605d-4aa0-b5ad-379f05fce885 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.130672 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.131308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.131583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.131853 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] PATH_INFO: `/v3/projects/d32617bdb5b94d468ecba14659b67c7e/users/eae7d8b1dca54688b3cc953f830d9007/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.146973 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d32617bdb5b94d468ecba14659b67c7e, user_id=eae7d8b1dca54688b3cc953f830d9007, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.149669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.239960 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1407/2800] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:06 2022] PUT /identity/v3/projects/d32617bdb5b94d468ecba14659b67c7e/users/eae7d8b1dca54688b3cc953f830d9007/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:06.303018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cc3ce-5b40-48e8-b8ee-a712b7ba3d66 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.303324 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cc3ce-5b40-48e8-b8ee-a712b7ba3d66 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.303594 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cc3ce-5b40-48e8-b8ee-a712b7ba3d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.331829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-a22cc3ce-5b40-48e8-b8ee-a712b7ba3d66 None None] MFA Rules not processed for user `eae7d8b1dca54688b3cc953f830d9007`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:06.332947 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-a22cc3ce-5b40-48e8-b8ee-a712b7ba3d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:06.394511 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1394/2801] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:06.398984 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.460677 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33491df9-2630-4693-9365-b7d9f43dd070 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.465382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.465972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.466251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.466557 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.467174 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.469059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.583826 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1408/2802] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:52:06 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:06.587961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cc3ce-5b40-48e8-b8ee-a712b7ba3d66 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.599343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cc3ce-5b40-48e8-b8ee-a712b7ba3d66 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.652586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.653178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.653456 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.653727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.654334 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.656251 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.719113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1395/2803] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:52:06 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:06.724039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.734516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343c9cc-454f-4a76-a944-2f2878a475bd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.739519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.740097 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.740372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.740638 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.741268 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.742860 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.753625 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1409/2804] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:06.757575 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.767938 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865d991-1970-4c6b-80e8-901d5d10e889 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.772527 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.773160 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.773439 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.773711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] PATH_INFO: `/v3/projects/5896c05cb81047aea5894908b9420e73/users/8f8eb16943a14f4dabc7db3340aac2fb/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.788711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5896c05cb81047aea5894908b9420e73, user_id=8f8eb16943a14f4dabc7db3340aac2fb, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.791361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.856536 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1396/2805] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:06 2022] PUT /identity/v3/projects/5896c05cb81047aea5894908b9420e73/users/8f8eb16943a14f4dabc7db3340aac2fb/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:06.860298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.865586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:06.870938 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ea11c-b2d2-4452-b912-9d5fb0206c57 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.927657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2f116-4bfc-41ed-acd0-32b8b4671ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.928263 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2f116-4bfc-41ed-acd0-32b8b4671ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.929812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2f116-4bfc-41ed-acd0-32b8b4671ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.929812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2f116-4bfc-41ed-acd0-32b8b4671ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.929812 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2f116-4bfc-41ed-acd0-32b8b4671ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.931114 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2f116-4bfc-41ed-acd0-32b8b4671ddc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.941535 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1410/2806] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:06.945855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:06.956730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47269-bad3-4776-abaa-2a6013baf241 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:06.961435 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:06.962073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.962373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.962669 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] PATH_INFO: `/v3/projects/5896c05cb81047aea5894908b9420e73/users/8f8eb16943a14f4dabc7db3340aac2fb/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:06.964608 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5896c05cb81047aea5894908b9420e73, user_id=8f8eb16943a14f4dabc7db3340aac2fb, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:06.967244 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:06.977959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1397/2807] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:06 2022] PUT /identity/v3/projects/5896c05cb81047aea5894908b9420e73/users/8f8eb16943a14f4dabc7db3340aac2fb/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:06.987751 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc5245-e76a-4c1a-9540-56c624f61f54 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:06.988037 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc5245-e76a-4c1a-9540-56c624f61f54 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:06.988341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc5245-e76a-4c1a-9540-56c624f61f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:07.018030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-03dc5245-e76a-4c1a-9540-56c624f61f54 None None] MFA Rules not processed for user `8f8eb16943a14f4dabc7db3340aac2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:07.019269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-03dc5245-e76a-4c1a-9540-56c624f61f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:07.086862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1411/2808] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:07.098054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:07.099748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:07.144927 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:07.177846 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d4dad-1ed1-4d40-9b79-c5cc7314fbb0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:07.181562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:07.182158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:07.182460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:07.182730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:07.193257 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:07.194783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:07.204062 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1398/2809] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:52:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:52:07.438863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc5245-e76a-4c1a-9540-56c624f61f54 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:07.448501 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc5245-e76a-4c1a-9540-56c624f61f54 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:07.455061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f2396-7eeb-4531-9c26-641ebcccfca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:07.455061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f2396-7eeb-4531-9c26-641ebcccfca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:07.455061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f2396-7eeb-4531-9c26-641ebcccfca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:07.455061 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f2396-7eeb-4531-9c26-641ebcccfca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:07.510228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f2396-7eeb-4531-9c26-641ebcccfca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:07.511689 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f2396-7eeb-4531-9c26-641ebcccfca2 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:07.520962 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1412/2810] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:52:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:52:17.225166 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=17 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:17.516546 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:18.910262 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:18.973179 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a035f-cdf2-4262-b376-cd397d520cfe service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:18.978528 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:18.979140 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:18.979412 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:18.979675 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] PATH_INFO: `/v3/users/eae7d8b1dca54688b3cc953f830d9007` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:18.980771 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae7d8b1dca54688b3cc953f830d9007)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:18.982763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:19.201194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae7d8b1dca54688b3cc953f830d9007'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:19.276288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae7d8b1dca54688b3cc953f830d9007'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:19.289014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae7d8b1dca54688b3cc953f830d9007'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:19.295950 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae7d8b1dca54688b3cc953f830d9007'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:19.296640 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae7d8b1dca54688b3cc953f830d9007'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:19.302499 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1399/2811] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:52:18 2022] DELETE /identity/v3/users/eae7d8b1dca54688b3cc953f830d9007 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:19.307290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:19.307782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:19.307885 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:19.326913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:19.328111 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:19.388116 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1413/2812] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:19.398302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:19.399745 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:19.454586 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370db4fa-8081-4bdf-9940-67e65e0fe121 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:19.458052 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:19.458667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:19.458942 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:19.459206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:19.468540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:19.469955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:19.480146 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1400/2813] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:52:19.886615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:19.888141 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:19.900353 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c098d39-cdcc-4aa0-a904-1e89c9ce0dba None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:19.904988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:19.905611 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:19.905888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:19.906159 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] PATH_INFO: `/v3/projects/d32617bdb5b94d468ecba14659b67c7e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:19.907326 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d32617bdb5b94d468ecba14659b67c7e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:19.909223 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:20.051959 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd32617bdb5b94d468ecba14659b67c7e', 'request_id': 'req-323be48d-2c15-4385-9001-6b5d9bd0b484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.113436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd32617bdb5b94d468ecba14659b67c7e', 'request_id': 'req-323be48d-2c15-4385-9001-6b5d9bd0b484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.121591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd32617bdb5b94d468ecba14659b67c7e', 'request_id': 'req-323be48d-2c15-4385-9001-6b5d9bd0b484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.132436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] The token cache is being invalidate because project d32617bdb5b94d468ecba14659b67c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:52:20.132737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.133373 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.134901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1414/2814] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:52:19 2022] DELETE /identity/v3/projects/d32617bdb5b94d468ecba14659b67c7e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:20.138893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:20.149288 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67c376-52ae-4693-bec8-cb84249fa9bb service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:20.167650 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:20.204894 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:20.205509 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:20.205788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:20.206060 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] PATH_INFO: `/v3/users/8f8eb16943a14f4dabc7db3340aac2fb` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:20.207215 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8eb16943a14f4dabc7db3340aac2fb)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:20.209136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:20.350193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8eb16943a14f4dabc7db3340aac2fb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.399374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8eb16943a14f4dabc7db3340aac2fb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.411353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8eb16943a14f4dabc7db3340aac2fb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.418429 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8eb16943a14f4dabc7db3340aac2fb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.419111 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-fd1ac0df-e298-4d31-9ce8-fb8d5fc4604f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8eb16943a14f4dabc7db3340aac2fb'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:20.424983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1401/2815] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:52:20 2022] DELETE /identity/v3/users/8f8eb16943a14f4dabc7db3340aac2fb => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:20.826228 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:20.827816 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:20.841172 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323be48d-2c15-4385-9001-6b5d9bd0b484 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:20.898277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:20.898857 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:20.899135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:20.899404 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] PATH_INFO: `/v3/projects/5896c05cb81047aea5894908b9420e73` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:20.900553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5896c05cb81047aea5894908b9420e73)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:20.902478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:20.981125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:20.997920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5896c05cb81047aea5894908b9420e73', 'request_id': 'req-ab7a45dc-3449-489f-a636-fdce3dcc623d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:21.062198 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5896c05cb81047aea5894908b9420e73', 'request_id': 'req-ab7a45dc-3449-489f-a636-fdce3dcc623d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:21.069100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5896c05cb81047aea5894908b9420e73', 'request_id': 'req-ab7a45dc-3449-489f-a636-fdce3dcc623d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:21.080820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] The token cache is being invalidate because project 5896c05cb81047aea5894908b9420e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:52:21.081133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:21.081771 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ab7a45dc-3449-489f-a636-fdce3dcc623d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:21.083269 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1415/2816] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:52:20 2022] DELETE /identity/v3/projects/5896c05cb81047aea5894908b9420e73 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:21.088538 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84004f-800c-4465-a672-693b9bb200c1 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.088864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84004f-800c-4465-a672-693b9bb200c1 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.089136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84004f-800c-4465-a672-693b9bb200c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.109616 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-6c84004f-800c-4465-a672-693b9bb200c1 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:21.110832 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-6c84004f-800c-4465-a672-693b9bb200c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:21.170376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1402/2817] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:21.180380 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79148849-dc86-4660-ba42-b95b83838728 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.180657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79148849-dc86-4660-ba42-b95b83838728 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.180924 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79148849-dc86-4660-ba42-b95b83838728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.201125 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-79148849-dc86-4660-ba42-b95b83838728 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:21.202399 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-79148849-dc86-4660-ba42-b95b83838728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:21.210424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1416/2818] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:21.214814 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84004f-800c-4465-a672-693b9bb200c1 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.224829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84004f-800c-4465-a672-693b9bb200c1 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.230079 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.230704 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.230984 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.231254 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.231926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:52:21.232224 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.233840 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:21.245943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1403/2819] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:52:21.250075 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79148849-dc86-4660-ba42-b95b83838728 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.259735 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79148849-dc86-4660-ba42-b95b83838728 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.264454 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.265040 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.265319 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.265589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.266215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.268163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:21.355142 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1417/2820] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:52:21 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:21.359265 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.369940 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14386bd7-5a52-4911-9132-6f24391bc8fc admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.394316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:21.423358 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.423878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.424151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.424419 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.425031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.426959 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:21.499480 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1404/2821] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:52:21 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:21.503572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.513934 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf16cb-6cb8-431d-ae70-34d9aefd6c8b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.518863 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.519457 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.519737 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.520006 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.520648 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.523030 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:21.533252 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1418/2822] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:21.537112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.547681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0d561-b7b0-4ed0-a0a6-5a024bb29cb7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.552431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e8953-fc87-4266-bc1b-f3fefaae30d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.553018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e8953-fc87-4266-bc1b-f3fefaae30d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.553295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e8953-fc87-4266-bc1b-f3fefaae30d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.553603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e8953-fc87-4266-bc1b-f3fefaae30d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.554236 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e8953-fc87-4266-bc1b-f3fefaae30d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.555857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e8953-fc87-4266-bc1b-f3fefaae30d7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:21.566552 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1405/2823] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:21.570649 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.580583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e5a25-2cc4-41f5-8014-d60174ab9018 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.585699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.586333 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.586643 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.586913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] PATH_INFO: `/v3/projects/3a179ac44a0947ff93cf8f0ce41c5c64/users/2ec08b2237e449e3b3e959fa72f97089/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.601594 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a179ac44a0947ff93cf8f0ce41c5c64, user_id=2ec08b2237e449e3b3e959fa72f97089, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.604251 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:21.679368 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1419/2824] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:21 2022] PUT /identity/v3/projects/3a179ac44a0947ff93cf8f0ce41c5c64/users/2ec08b2237e449e3b3e959fa72f97089/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:21.689732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0a6ea-8296-4d8c-9b4f-2148dfcb9292 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.690014 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0a6ea-8296-4d8c-9b4f-2148dfcb9292 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.690283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0a6ea-8296-4d8c-9b4f-2148dfcb9292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.717707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-8ea0a6ea-8296-4d8c-9b4f-2148dfcb9292 None None] MFA Rules not processed for user `2ec08b2237e449e3b3e959fa72f97089`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:21.718930 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8ea0a6ea-8296-4d8c-9b4f-2148dfcb9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:21.780596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1406/2825] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:52:21.784990 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.846972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3608d-3b8e-4010-b88a-2192c2c6f440 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.851862 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.852455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.852732 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.853004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.853622 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.855586 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:21.923624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1420/2826] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:52:21 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:21.927759 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0a6ea-8296-4d8c-9b4f-2148dfcb9292 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:21.938913 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0a6ea-8296-4d8c-9b4f-2148dfcb9292 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:21.995902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:21.996495 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:21.996769 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:21.997034 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:21.997651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:21.999576 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:22.099869 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1407/2827] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:52:21 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:22.103920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:22.114375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e6496-0f2b-4ff5-a7df-9e3749f3f206 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:22.119014 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:22.119593 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:22.119865 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:22.120129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:22.120756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:22.122374 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:22.134407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1421/2828] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:22.138423 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:22.149230 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28f1f0-0c27-45e5-8c43-edb7e1f6618f admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:22.154129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:22.154809 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:22.155089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:22.155356 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] PATH_INFO: `/v3/projects/c726340389db4712a81d31ed479d9e91/users/70318311a3ef4e85b404d1a550888a52/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:22.170345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c726340389db4712a81d31ed479d9e91, user_id=70318311a3ef4e85b404d1a550888a52, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:22.173093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:22.239416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1408/2829] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:22 2022] PUT /identity/v3/projects/c726340389db4712a81d31ed479d9e91/users/70318311a3ef4e85b404d1a550888a52/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:22.244099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:22.254595 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372ac8-3dd5-4638-8486-12521d4851b1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:22.275983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:22.313004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59088-9a08-4e67-b575-8418dfe3972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:22.313786 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59088-9a08-4e67-b575-8418dfe3972d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:22.314447 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59088-9a08-4e67-b575-8418dfe3972d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:22.314774 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59088-9a08-4e67-b575-8418dfe3972d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:22.315437 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b59088-9a08-4e67-b575-8418dfe3972d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:22.317050 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b59088-9a08-4e67-b575-8418dfe3972d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:22.328051 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1422/2830] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:22.332259 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:22.338013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:22.343333 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de971b8a-a6e9-45fe-a7e7-f5286fba2f17 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:22.347970 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:22.348603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:22.348896 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:22.349151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] PATH_INFO: `/v3/projects/c726340389db4712a81d31ed479d9e91/users/70318311a3ef4e85b404d1a550888a52/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:22.351217 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c726340389db4712a81d31ed479d9e91, user_id=70318311a3ef4e85b404d1a550888a52, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:22.353898 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:22.363911 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1409/2831] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:22 2022] PUT /identity/v3/projects/c726340389db4712a81d31ed479d9e91/users/70318311a3ef4e85b404d1a550888a52/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:22.374855 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47d5c1-5fb2-40db-94d5-b143a3699acf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:22.375140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47d5c1-5fb2-40db-94d5-b143a3699acf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:22.375410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47d5c1-5fb2-40db-94d5-b143a3699acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:22.403266 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8e47d5c1-5fb2-40db-94d5-b143a3699acf None None] MFA Rules not processed for user `70318311a3ef4e85b404d1a550888a52`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:22.404529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e47d5c1-5fb2-40db-94d5-b143a3699acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:22.476846 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1423/2832] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:22.488040 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:22.489801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:22.546556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91975050-1808-4985-885e-05d2e6d7fbb0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:22.550209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006fac3-a941-4bb8-91ec-620d7ae8e514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:22.550829 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006fac3-a941-4bb8-91ec-620d7ae8e514 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:22.551107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006fac3-a941-4bb8-91ec-620d7ae8e514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:22.551374 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006fac3-a941-4bb8-91ec-620d7ae8e514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:22.609343 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006fac3-a941-4bb8-91ec-620d7ae8e514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:22.610806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006fac3-a941-4bb8-91ec-620d7ae8e514 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:22.620842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1410/2833] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:52:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:52:25.670149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47d5c1-5fb2-40db-94d5-b143a3699acf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:25.732229 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47d5c1-5fb2-40db-94d5-b143a3699acf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:25.736770 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:25.737382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:25.737658 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:25.737931 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] PATH_INFO: `/v3/users/2ec08b2237e449e3b3e959fa72f97089` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:25.739147 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec08b2237e449e3b3e959fa72f97089)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:25.741808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:25.927407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec08b2237e449e3b3e959fa72f97089'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:25.929783 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:26.145311 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec08b2237e449e3b3e959fa72f97089'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:26.160087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec08b2237e449e3b3e959fa72f97089'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:26.160777 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec08b2237e449e3b3e959fa72f97089'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:26.161377 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec08b2237e449e3b3e959fa72f97089'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:26.174551 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1424/2834] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:52:25 2022] DELETE /identity/v3/users/2ec08b2237e449e3b3e959fa72f97089 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:26.179098 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:26.179596 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:26.179707 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:26.199295 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:26.200472 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:26.259585 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1411/2835] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:26.269387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:26.270943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:26.327958 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e0a51-2fec-4d44-9b0a-67f5ef774d19 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:26.331587 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:26.332178 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:26.332455 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:26.332724 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:26.341720 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:26.343180 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:26.353256 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1425/2836] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:52:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:52:26.737850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:26.739278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:26.753221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56f94a-aff8-4ae8-af2d-e93e514c8ae8 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:26.758678 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:26.761027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:26.761027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:26.761027 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] PATH_INFO: `/v3/projects/3a179ac44a0947ff93cf8f0ce41c5c64` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:26.762279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a179ac44a0947ff93cf8f0ce41c5c64)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:26.764201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:26.890539 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:26.897352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a179ac44a0947ff93cf8f0ce41c5c64', 'request_id': 'req-0da82859-2210-4d6d-9090-e84c650f971b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:26.903396 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a179ac44a0947ff93cf8f0ce41c5c64', 'request_id': 'req-0da82859-2210-4d6d-9090-e84c650f971b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:26.993323 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a179ac44a0947ff93cf8f0ce41c5c64', 'request_id': 'req-0da82859-2210-4d6d-9090-e84c650f971b'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.004390 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] The token cache is being invalidate because project 3a179ac44a0947ff93cf8f0ce41c5c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:52:27.004876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.005360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.007631 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1412/2837] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:52:26 2022] DELETE /identity/v3/projects/3a179ac44a0947ff93cf8f0ce41c5c64 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:27.011748 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:27.021767 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135706-3e65-478f-b314-9b4c10650456 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:27.080129 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:27.080746 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:27.081024 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:27.081295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] PATH_INFO: `/v3/users/70318311a3ef4e85b404d1a550888a52` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:27.082465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70318311a3ef4e85b404d1a550888a52)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:27.084383 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:27.223485 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70318311a3ef4e85b404d1a550888a52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.290615 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70318311a3ef4e85b404d1a550888a52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.301782 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:27.302940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70318311a3ef4e85b404d1a550888a52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.304212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70318311a3ef4e85b404d1a550888a52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.304965 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b4864189-6f5b-4545-b843-430fc5ca9bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70318311a3ef4e85b404d1a550888a52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.316963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1426/2838] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:52:27 2022] DELETE /identity/v3/users/70318311a3ef4e85b404d1a550888a52 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:27.726967 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:27.739286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:27.742892 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da82859-2210-4d6d-9090-e84c650f971b admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:27.800730 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:27.801351 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:27.801625 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:27.801889 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] PATH_INFO: `/v3/projects/c726340389db4712a81d31ed479d9e91` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:27.803147 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c726340389db4712a81d31ed479d9e91)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:27.805614 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:27.928831 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c726340389db4712a81d31ed479d9e91', 'request_id': 'req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.935116 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c726340389db4712a81d31ed479d9e91', 'request_id': 'req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:27.993286 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c726340389db4712a81d31ed479d9e91', 'request_id': 'req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:28.004424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] The token cache is being invalidate because project c726340389db4712a81d31ed479d9e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:52:28.004711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:28.005405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-ec035b51-11f0-4ac3-bb5b-bcf18c8d7b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:52:28.007547 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1413/2839] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:52:27 2022] DELETE /identity/v3/projects/c726340389db4712a81d31ed479d9e91 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:28.012614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fad03-7ebd-40d5-b58c-dc8d32790901 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.013094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fad03-7ebd-40d5-b58c-dc8d32790901 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.013207 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fad03-7ebd-40d5-b58c-dc8d32790901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.033396 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-215fad03-7ebd-40d5-b58c-dc8d32790901 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:28.034634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-215fad03-7ebd-40d5-b58c-dc8d32790901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:28.095733 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1427/2840] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:28.105018 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f0de6-9e40-47d3-adbb-7d7a629708ab None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.105304 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f0de6-9e40-47d3-adbb-7d7a629708ab None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.105578 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f0de6-9e40-47d3-adbb-7d7a629708ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.125823 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-303f0de6-9e40-47d3-adbb-7d7a629708ab None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:28.127045 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-303f0de6-9e40-47d3-adbb-7d7a629708ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:28.135064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1414/2841] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:28.139205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fad03-7ebd-40d5-b58c-dc8d32790901 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:28.148929 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fad03-7ebd-40d5-b58c-dc8d32790901 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:28.153520 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:28.154108 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.154412 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.154684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.155322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:52:28.155621 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:28.157290 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:28.169226 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1428/2842] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:52:28.173347 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f0de6-9e40-47d3-adbb-7d7a629708ab None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:28.183393 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f0de6-9e40-47d3-adbb-7d7a629708ab None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:28.188151 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:28.188727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.189020 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.189283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.189886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:28.191794 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:28.283935 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1415/2843] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:52:28 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:52:28.289004 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:28.298187 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a0b8-2119-4a2e-b61c-d799bcf19e15 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:28.354193 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:28.354809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.355085 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.355355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.355972 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:28.357871 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:28.424407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1429/2844] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:52:28 2022] POST /identity/v3/users => generated 283 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:52:28.429158 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:28.439858 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7abb9-d4fb-4132-9487-e45bafafa0b0 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:28.445135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:28.445718 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.445997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.446289 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.446962 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:28.448553 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:28.458278 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1416/2845] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:28.462080 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:28.472166 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad83ef1-b47c-48d8-95d0-91e82db52c9a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:28.476968 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c018b70-0cd2-4309-b168-27801c001527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:28.477543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018b70-0cd2-4309-b168-27801c001527 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.477820 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018b70-0cd2-4309-b168-27801c001527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.478088 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018b70-0cd2-4309-b168-27801c001527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.478754 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c018b70-0cd2-4309-b168-27801c001527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:28.480321 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c018b70-0cd2-4309-b168-27801c001527 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:28.492756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1430/2846] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:52:28.496646 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:28.506623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fbde6-5263-43ed-a366-83c87c9b2adf admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:28.511887 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:28.512519 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.512795 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.513064 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] PATH_INFO: `/v3/projects/da5eca79501543e4a44da65c8f029e21/users/6b40e051f6ba46b896d81ab8ab220921/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.516026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:28.528404 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5eca79501543e4a44da65c8f029e21, user_id=6b40e051f6ba46b896d81ab8ab220921, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:28.531135 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:28.624633 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1417/2847] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:52:28 2022] PUT /identity/v3/projects/da5eca79501543e4a44da65c8f029e21/users/6b40e051f6ba46b896d81ab8ab220921/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:52:28.634992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7b2ae-2f41-4eaf-8a74-22800c9f8704 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.635310 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7b2ae-2f41-4eaf-8a74-22800c9f8704 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.635579 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7b2ae-2f41-4eaf-8a74-22800c9f8704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.662543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-95d7b2ae-2f41-4eaf-8a74-22800c9f8704 None None] MFA Rules not processed for user `6b40e051f6ba46b896d81ab8ab220921`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:52:28.664516 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-95d7b2ae-2f41-4eaf-8a74-22800c9f8704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:28.705316 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:28.728859 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1431/2848] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:52:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:52:28.740504 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:52:28.744734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:52:28.800360 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919b5c8-469d-44ac-b5d8-9cfd3b3f3686 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:52:28.804702 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429158be-8a0b-49fc-a77c-58739e074653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:52:28.807842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429158be-8a0b-49fc-a77c-58739e074653 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:52:28.807842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429158be-8a0b-49fc-a77c-58739e074653 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:52:28.807842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429158be-8a0b-49fc-a77c-58739e074653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:52:28.814380 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429158be-8a0b-49fc-a77c-58739e074653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:52:28.815810 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429158be-8a0b-49fc-a77c-58739e074653 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:52:28.824661 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1418/2849] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:52:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:52:38.796267 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:52:38.820667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:13.690583 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7b2ae-2f41-4eaf-8a74-22800c9f8704 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:13.752899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7b2ae-2f41-4eaf-8a74-22800c9f8704 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:13.757408 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:13.758025 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:13.758302 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:13.758600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] PATH_INFO: `/v3/users/6b40e051f6ba46b896d81ab8ab220921` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:13.759706 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b40e051f6ba46b896d81ab8ab220921)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:13.762103 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:13.940725 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b40e051f6ba46b896d81ab8ab220921'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.000335 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b40e051f6ba46b896d81ab8ab220921'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.013151 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b40e051f6ba46b896d81ab8ab220921'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.013789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b40e051f6ba46b896d81ab8ab220921'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.014424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b40e051f6ba46b896d81ab8ab220921'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.026484 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1432/2850] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:53:13 2022] DELETE /identity/v3/users/6b40e051f6ba46b896d81ab8ab220921 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:53:14.030943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:14.031450 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:14.031574 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:14.050012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:14.051340 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:14.111037 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1419/2851] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:14.121330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:14.122759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:14.178983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3543e8-41de-4375-aca6-df8f3d2a645d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:14.182598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc007508-2ca8-4249-981e-d99f7b060374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:14.183716 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc007508-2ca8-4249-981e-d99f7b060374 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:14.183982 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc007508-2ca8-4249-981e-d99f7b060374 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:14.184248 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc007508-2ca8-4249-981e-d99f7b060374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:14.194163 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc007508-2ca8-4249-981e-d99f7b060374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:14.195669 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc007508-2ca8-4249-981e-d99f7b060374 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:14.205974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1433/2852] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:53:14.611788 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:14.614394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:14.625876 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baffcf-d6b7-46ca-9ec5-d93104907a89 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:14.630556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:14.631176 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:14.631449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:14.631715 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] PATH_INFO: `/v3/projects/da5eca79501543e4a44da65c8f029e21` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:14.632862 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5eca79501543e4a44da65c8f029e21)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:14.634763 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:14.718857 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5eca79501543e4a44da65c8f029e21', 'request_id': 'req-2d32e24d-597a-4bb7-8097-df5beaedc680'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.726414 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5eca79501543e4a44da65c8f029e21', 'request_id': 'req-2d32e24d-597a-4bb7-8097-df5beaedc680'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.775271 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5eca79501543e4a44da65c8f029e21', 'request_id': 'req-2d32e24d-597a-4bb7-8097-df5beaedc680'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.786732 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] The token cache is being invalidate because project da5eca79501543e4a44da65c8f029e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:53:14.787026 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.787664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-2d32e24d-597a-4bb7-8097-df5beaedc680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:14.789173 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1420/2853] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:53:14 2022] DELETE /identity/v3/projects/da5eca79501543e4a44da65c8f029e21 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:53:14.796369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee724a2-9027-42a4-b034-d892233618c0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:14.796870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee724a2-9027-42a4-b034-d892233618c0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:14.796987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee724a2-9027-42a4-b034-d892233618c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:14.816632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8ee724a2-9027-42a4-b034-d892233618c0 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:14.817827 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8ee724a2-9027-42a4-b034-d892233618c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:14.834553 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:14.877382 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1434/2854] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:14.885028 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1a508-0e53-4e13-84d6-0c328e449a85 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:14.885314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1a508-0e53-4e13-84d6-0c328e449a85 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:14.885591 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1a508-0e53-4e13-84d6-0c328e449a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:14.906693 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f1c1a508-0e53-4e13-84d6-0c328e449a85 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:14.907903 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f1c1a508-0e53-4e13-84d6-0c328e449a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:14.915886 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1421/2855] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:14.920809 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee724a2-9027-42a4-b034-d892233618c0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:14.930925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee724a2-9027-42a4-b034-d892233618c0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:14.935892 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:14.936741 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:14.937022 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:14.937298 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:14.937940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:53:14.938244 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:14.939913 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:14.951726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1435/2856] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:53:14.956364 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1a508-0e53-4e13-84d6-0c328e449a85 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:14.966783 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1a508-0e53-4e13-84d6-0c328e449a85 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:14.971645 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:14.972183 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:14.972460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:14.972728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] PATH_INFO: `/v3/projects` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:14.973352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:14.975315 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:15.063119 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1422/2857] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:53:14 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:53:15.067259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:15.078295 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3de917-8773-4364-bc35-0c0295a2b4fa admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:15.141300 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:15.141921 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:15.142253 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:15.142561 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:15.143317 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:15.145260 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:15.250204 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1436/2858] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:53:15 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:53:15.254314 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:15.265500 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c013d4a-d890-44f8-a0bc-fa700ce4b62e admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:15.269978 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:15.270961 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:15.271237 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:15.271502 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:15.272134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:15.273711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:15.284522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1423/2859] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:53:15.288358 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:15.298153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856662-1946-437e-862a-ec8fabc2dc8f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:15.302822 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9e5af-c884-414e-b665-7ad00b6d2db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:15.303400 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e5af-c884-414e-b665-7ad00b6d2db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:15.303674 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e5af-c884-414e-b665-7ad00b6d2db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:15.303943 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9e5af-c884-414e-b665-7ad00b6d2db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:15.304572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9e5af-c884-414e-b665-7ad00b6d2db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:15.306138 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9e5af-c884-414e-b665-7ad00b6d2db6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:15.316645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1437/2860] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:53:15.320613 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:15.331148 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4de53-7a6f-4971-96c8-6ae041c3caf5 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:15.335822 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:15.336449 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:15.336728 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:15.336996 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] PATH_INFO: `/v3/projects/370376db13ed46e1bfbff9a5741f87ac/users/2ac4d7cd8de1449ab0a3b143971a2616/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:15.352522 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=370376db13ed46e1bfbff9a5741f87ac, user_id=2ac4d7cd8de1449ab0a3b143971a2616, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:15.355341 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:15.446366 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1424/2861] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:53:15 2022] PUT /identity/v3/projects/370376db13ed46e1bfbff9a5741f87ac/users/2ac4d7cd8de1449ab0a3b143971a2616/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:53:15.457372 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fe05c-51f3-43c9-9f87-f86e27448bf8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:15.457655 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fe05c-51f3-43c9-9f87-f86e27448bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:15.457926 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fe05c-51f3-43c9-9f87-f86e27448bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:15.484988 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d82fe05c-51f3-43c9-9f87-f86e27448bf8 None None] MFA Rules not processed for user `2ac4d7cd8de1449ab0a3b143971a2616`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:15.486190 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d82fe05c-51f3-43c9-9f87-f86e27448bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:15.548757 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1438/2862] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:15.559751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:15.561700 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:15.584058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:15.617850 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6a44-b2c0-49cb-a634-da1c9220b16d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:15.622317 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:15.624212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:15.624212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:15.624212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:15.631696 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:15.633124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:15.642870 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1425/2863] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:53:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:53:17.421265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:17.421265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:17.421265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:17.421265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] MFA Rules not processed for user `2ac4d7cd8de1449ab0a3b143971a2616`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:17.421265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:17.421265 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1439/2864] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:17.442042 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:17.443446 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:17.521189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801e569-c2eb-41ec-b517-4db7e324bdd6 service cinder] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:17.524801 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a905c35-399c-4340-baab-bbee9e9d2abc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38206a70a8b24cffabac622175335154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:17.527283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a905c35-399c-4340-baab-bbee9e9d2abc service glance] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:17.527283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a905c35-399c-4340-baab-bbee9e9d2abc service glance] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:17.527283 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a905c35-399c-4340-baab-bbee9e9d2abc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:17.527664 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-1a905c35-399c-4340-baab-bbee9e9d2abc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:17.615089 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a905c35-399c-4340-baab-bbee9e9d2abc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:17.616430 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a905c35-399c-4340-baab-bbee9e9d2abc service glance] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:17.636112 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1426/2865] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:53:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:53:18.420106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:18.421543 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:18.435062 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:18.480987 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8591e60-92a8-469b-bc64-5109648cad05 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:18.484842 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:18.485397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:18.485687 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:18.485946 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:18.487876 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:18.501983 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:18.503507 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:18.516473 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1440/2866] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:53:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:53:27.633362 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:28.577708 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=21 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:36.535703 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989395f5-4c81-40f9-92f6-8efec6474d02 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:36.536170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989395f5-4c81-40f9-92f6-8efec6474d02 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:36.536276 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989395f5-4c81-40f9-92f6-8efec6474d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:36.595571 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-989395f5-4c81-40f9-92f6-8efec6474d02 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:36.596773 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-989395f5-4c81-40f9-92f6-8efec6474d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:36.659923 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1427/2867] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:53:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:36.786278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:36.787740 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:36.879140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f3402-0f58-4df6-adde-9d95e8b33267 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:36.884245 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:36.884838 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:36.885119 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:36.885393 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:36.893660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:36.895262 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:36.903345 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1441/2868] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:53:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:53:37.673110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737d416-6889-4a62-bab8-a85d7ce40824 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:37.673588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737d416-6889-4a62-bab8-a85d7ce40824 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:37.673712 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737d416-6889-4a62-bab8-a85d7ce40824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:37.693029 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-4737d416-6889-4a62-bab8-a85d7ce40824 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:37.694252 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-4737d416-6889-4a62-bab8-a85d7ce40824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:37.702141 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1428/2869] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 02:53:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:37.714233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:37.725537 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cb467-4d69-4512-b9ba-ac45b7a4c8fc service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:37.730565 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:37.731191 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:37.731462 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:37.731726 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:37.739645 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:37.741192 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:37.749078 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1442/2870] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:53:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:53:38.332612 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeaf8-a87a-4c3e-8941-af158b441a9c None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:38.337266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeaf8-a87a-4c3e-8941-af158b441a9c None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:38.337266 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeaf8-a87a-4c3e-8941-af158b441a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:38.352005 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f05eeaf8-a87a-4c3e-8941-af158b441a9c None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:38.353194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f05eeaf8-a87a-4c3e-8941-af158b441a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:38.361070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1429/2871] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 02:53:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:38.373654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:38.384367 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acb9d-d3c3-47c3-a08c-c303cfa5747a service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:38.389804 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:38.390436 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:38.390718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:38.390989 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:38.398994 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:38.400555 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:38.409067 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1443/2872] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:53:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:53:38.999170 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1335d-f185-46fb-8ebe-792f292e4ec3 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:38.999639 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1335d-f185-46fb-8ebe-792f292e4ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:38.999738 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1335d-f185-46fb-8ebe-792f292e4ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:39.018919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-aca1335d-f185-46fb-8ebe-792f292e4ec3 None None] MFA Rules not processed for user `6aa66d4b5bad476183509a4e00c56967`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:39.020118 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-aca1335d-f185-46fb-8ebe-792f292e4ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:39.027605 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1430/2873] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 02:53:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:39.039888 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:39.051215 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff434-63e5-42d4-9df0-a5ba48b37243 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:39.056369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbec379-7619-4458-a337-b114350ac620 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '481259c751234a5a936a8b0660eb8cac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:39.056944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbec379-7619-4458-a337-b114350ac620 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:39.057218 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbec379-7619-4458-a337-b114350ac620 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:39.057488 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbec379-7619-4458-a337-b114350ac620 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:39.066379 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbec379-7619-4458-a337-b114350ac620 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:39.068053 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbec379-7619-4458-a337-b114350ac620 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:39.077018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1444/2874] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 02:53:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:53:41.677300 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1335d-f185-46fb-8ebe-792f292e4ec3 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:41.737494 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1335d-f185-46fb-8ebe-792f292e4ec3 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:41.741919 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:41.742550 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:41.742825 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:41.743093 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] PATH_INFO: `/v3/users/2ac4d7cd8de1449ab0a3b143971a2616` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:41.744201 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac4d7cd8de1449ab0a3b143971a2616)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:41.746685 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:42.004312 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:42.005897 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac4d7cd8de1449ab0a3b143971a2616'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.088353 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac4d7cd8de1449ab0a3b143971a2616'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.101828 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac4d7cd8de1449ab0a3b143971a2616'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.109477 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac4d7cd8de1449ab0a3b143971a2616'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.110161 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac4d7cd8de1449ab0a3b143971a2616'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.120422 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1431/2875] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 02:53:41 2022] DELETE /identity/v3/users/2ac4d7cd8de1449ab0a3b143971a2616 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:53:42.120591 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:42.121699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:42.121699 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:42.140975 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:42.142185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:42.208936 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1445/2876] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:42.218925 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:42.223408 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:42.276778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384f555-0a05-4bf7-b213-6673e3a4d087 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:42.280546 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810746e-f942-4b47-a861-759cdbd7b3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:42.281143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810746e-f942-4b47-a861-759cdbd7b3ba service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:42.281415 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810746e-f942-4b47-a861-759cdbd7b3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:42.281681 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810746e-f942-4b47-a861-759cdbd7b3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:42.290926 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810746e-f942-4b47-a861-759cdbd7b3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:42.292359 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810746e-f942-4b47-a861-759cdbd7b3ba service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:42.302436 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1432/2877] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:53:42.559729 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:42.561519 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:42.578049 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1dedf-e377-4445-916d-23f522422c79 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:42.583277 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:42.583916 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:42.584194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:42.584465 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] PATH_INFO: `/v3/projects/370376db13ed46e1bfbff9a5741f87ac` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:42.585778 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=370376db13ed46e1bfbff9a5741f87ac)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:42.587807 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:42.730496 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:42.750394 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370376db13ed46e1bfbff9a5741f87ac', 'request_id': 'req-8601b4da-9a23-4c0f-8680-53dd4182fd08'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.906093 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370376db13ed46e1bfbff9a5741f87ac', 'request_id': 'req-8601b4da-9a23-4c0f-8680-53dd4182fd08'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.913788 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370376db13ed46e1bfbff9a5741f87ac', 'request_id': 'req-8601b4da-9a23-4c0f-8680-53dd4182fd08'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.925060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] The token cache is being invalidate because project 370376db13ed46e1bfbff9a5741f87ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:53:42.925397 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.926000 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8601b4da-9a23-4c0f-8680-53dd4182fd08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:53:42.927590 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1446/2878] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 02:53:42 2022] DELETE /identity/v3/projects/370376db13ed46e1bfbff9a5741f87ac => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:53:42.932524 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65a66d-37e9-4953-91ab-11d9b7191148 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:42.932997 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65a66d-37e9-4953-91ab-11d9b7191148 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:42.933115 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65a66d-37e9-4953-91ab-11d9b7191148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:42.958067 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-ea65a66d-37e9-4953-91ab-11d9b7191148 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:42.959373 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-ea65a66d-37e9-4953-91ab-11d9b7191148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:43.022463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1433/2879] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:43.031450 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336deb4c-3ba4-48e5-896d-d2a0dc34e5f5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.031742 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336deb4c-3ba4-48e5-896d-d2a0dc34e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.032020 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336deb4c-3ba4-48e5-896d-d2a0dc34e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.052410 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-336deb4c-3ba4-48e5-896d-d2a0dc34e5f5 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:43.053657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-336deb4c-3ba4-48e5-896d-d2a0dc34e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:43.062518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1447/2880] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:43.066751 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65a66d-37e9-4953-91ab-11d9b7191148 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:43.077602 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65a66d-37e9-4953-91ab-11d9b7191148 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:43.082302 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:43.082937 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.083218 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.083493 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.084136 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80267) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:53:43.084438 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:43.086031 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:43.098530 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1434/2881] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 02:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:53:43.102632 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336deb4c-3ba4-48e5-896d-d2a0dc34e5f5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:43.112246 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336deb4c-3ba4-48e5-896d-d2a0dc34e5f5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:43.116935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:43.117526 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.117808 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.118077 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.118756 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:43.120644 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:43.224624 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1448/2882] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:53:43 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:53:43.228705 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:43.240562 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd70c93-f593-4ff5-bd23-6bacd1723e46 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:43.287918 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:43.297458 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:43.298054 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.299081 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.299431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.300055 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:43.301949 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:43.374733 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1435/2883] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:53:43 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:53:43.378895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:43.389780 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59e379-7992-4a32-a4ac-240811127830 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:43.394456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:43.395035 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.395308 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.395573 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.396205 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:43.397813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:43.408293 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1449/2884] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:53:43.416864 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:43.422948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c6758-3a5d-40cb-9e58-668e14187327 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:43.427603 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fe11e-b69a-4894-9aee-e1412fb3ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:43.428206 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fe11e-b69a-4894-9aee-e1412fb3ecf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.428491 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fe11e-b69a-4894-9aee-e1412fb3ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.428766 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fe11e-b69a-4894-9aee-e1412fb3ecf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.429399 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fe11e-b69a-4894-9aee-e1412fb3ecf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:43.431094 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fe11e-b69a-4894-9aee-e1412fb3ecf2 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:43.443320 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1436/2885] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:53:43.447323 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:43.458134 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825a99b-a3ad-439f-be58-79a8c58bcfc7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:43.462764 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:43.463386 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.463661 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.463927 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] PATH_INFO: `/v3/projects/cdf9c8d3eace468d852b873213f392f4/users/74a5ed7c176f44849677acf29c330e97/roles/34c3a19b7c7f45d1904aad07a5cec5b1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.479018 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf9c8d3eace468d852b873213f392f4, user_id=74a5ed7c176f44849677acf29c330e97, role_id=34c3a19b7c7f45d1904aad07a5cec5b1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:43.481908 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:43.571135 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1450/2886] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:53:43 2022] PUT /identity/v3/projects/cdf9c8d3eace468d852b873213f392f4/users/74a5ed7c176f44849677acf29c330e97/roles/34c3a19b7c7f45d1904aad07a5cec5b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:53:43.576955 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268d3a5-4931-438b-8c95-d7e389d77495 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.577238 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268d3a5-4931-438b-8c95-d7e389d77495 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.577537 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268d3a5-4931-438b-8c95-d7e389d77495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.597038 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-0268d3a5-4931-438b-8c95-d7e389d77495 None None] MFA Rules not processed for user `c5565152ee164c0da536fef7f9cd3d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:43.598231 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-0268d3a5-4931-438b-8c95-d7e389d77495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:43.655595 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1437/2887] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:43.665278 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:43.666949 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:43.723140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175d295-b1db-4e93-83b7-f367cb69daaa admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:43.726759 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:43.727339 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:43.727612 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:43.727877 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:43.736815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:43.738153 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:43.747351 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1451/2888] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:53:49.060615 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0da8-af23-4161-a94c-83ba1e541862 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:49.061134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0da8-af23-4161-a94c-83ba1e541862 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:49.061134 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d0da8-af23-4161-a94c-83ba1e541862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:49.102322 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-f98d0da8-af23-4161-a94c-83ba1e541862 None None] MFA Rules not processed for user `74a5ed7c176f44849677acf29c330e97`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:49.103806 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f98d0da8-af23-4161-a94c-83ba1e541862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:49.170818 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1438/2889] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:49.175341 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:49.185855 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26daf25-2d8d-4531-970d-73ba3a97509f service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:49.190572 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:49.191158 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:49.191434 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:49.191703 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:49.192322 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:49.194242 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:49.275294 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1452/2890] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 02:53:49 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:53:49.279413 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d0da8-af23-4161-a94c-83ba1e541862 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:49.290540 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d0da8-af23-4161-a94c-83ba1e541862 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:49.346966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:49.347568 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:49.347844 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:49.348113 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] PATH_INFO: `/v3/users` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:49.348729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:49.350680 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:49.359361 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:49.447802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1439/2891] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 02:53:49 2022] POST /identity/v3/users => generated 296 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:53:49.451212 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:49.467634 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ecb45-d4b1-45e0-a8e1-6114f0e64e62 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:49.472895 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:49.473598 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:49.473944 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:49.474371 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:49.478784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:49.478784 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:49.489209 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1453/2892] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:53:49.493129 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:49.507729 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae7c2f-d73d-4035-a894-68629936c4fa admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:49.510787 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:49.511454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:49.511785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:49.512012 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] PATH_INFO: `/v3/projects/90c196e44a4e40a49af4b5e27e3cceb1/users/5a998830266e4440869f918cb78f4221/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:49.529199 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c196e44a4e40a49af4b5e27e3cceb1, user_id=5a998830266e4440869f918cb78f4221, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:49.531842 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:49.608091 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1440/2893] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:53:49 2022] PUT /identity/v3/projects/90c196e44a4e40a49af4b5e27e3cceb1/users/5a998830266e4440869f918cb78f4221/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:53:49.612600 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:49.624940 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be869bf2-2e3e-4de1-8aed-4a116ce9962e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:49.642678 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:53:49.682164 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efdced-ffe8-4730-8f64-247f01b98f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:49.682969 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efdced-ffe8-4730-8f64-247f01b98f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:49.683140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efdced-ffe8-4730-8f64-247f01b98f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:49.684048 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efdced-ffe8-4730-8f64-247f01b98f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:49.684745 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efdced-ffe8-4730-8f64-247f01b98f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:49.686869 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efdced-ffe8-4730-8f64-247f01b98f87 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:49.697424 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1454/2894] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 02:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:53:49.701431 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:49.713765 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774822-8119-44d7-a75f-e894657c792d admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:49.718952 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:49.719609 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:49.719877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:49.720143 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] PATH_INFO: `/v3/projects/90c196e44a4e40a49af4b5e27e3cceb1/users/5a998830266e4440869f918cb78f4221/roles/1b9fb565648c49388166fe130739891f` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:49.722088 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c196e44a4e40a49af4b5e27e3cceb1, user_id=5a998830266e4440869f918cb78f4221, role_id=1b9fb565648c49388166fe130739891f)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:49.724753 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:49.735983 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1441/2895] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 02:53:49 2022] PUT /identity/v3/projects/90c196e44a4e40a49af4b5e27e3cceb1/users/5a998830266e4440869f918cb78f4221/roles/1b9fb565648c49388166fe130739891f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:53:55.169571 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f68d4-b0cc-4b46-b7a0-6b6af732a5a5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:55.170058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f68d4-b0cc-4b46-b7a0-6b6af732a5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:55.170185 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f68d4-b0cc-4b46-b7a0-6b6af732a5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:55.205449 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9f2f68d4-b0cc-4b46-b7a0-6b6af732a5a5 None None] MFA Rules not processed for user `5a998830266e4440869f918cb78f4221`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:53:55.207384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9f2f68d4-b0cc-4b46-b7a0-6b6af732a5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:55.278980 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1455/2896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:53:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:53:55.294985 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:53:55.296785 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:53:55.372110 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406f58-5b57-4ed3-91f1-4a48966be799 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:53:55.377659 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:53:55.378268 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:53:55.378582 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:53:55.378855 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:53:55.444076 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:53:55.445711 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:53:55.456070 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1442/2897] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:53:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:54:05.275518 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=54 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:54:05.451652 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:INSERT=4,keystone:SELECT=73 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:54:18.837184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fbb12-c596-4034-a24a-e3732df48336 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:54:18.837718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fbb12-c596-4034-a24a-e3732df48336 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:54:18.837718 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fbb12-c596-4034-a24a-e3732df48336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:54:18.857276 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-681fbb12-c596-4034-a24a-e3732df48336 None None] MFA Rules not processed for user `74a5ed7c176f44849677acf29c330e97`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:54:18.858495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-681fbb12-c596-4034-a24a-e3732df48336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:54:18.865257 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1456/2898] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:54:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:54:19.180194 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:54:19.235021 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db05e20-54c6-4ff8-9b4b-32b0e51a53a5 service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:54:19.238579 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:54:19.239165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:54:19.239440 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:54:19.239758 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:54:19.241402 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:54:19.251561 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:54:19.253051 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:54:19.261893 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1443/2899] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:54:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:54:23.062330 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fbb12-c596-4034-a24a-e3732df48336 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:54:23.073474 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fbb12-c596-4034-a24a-e3732df48336 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:54:23.077509 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:54:23.078387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:54:23.078387 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:54:23.078666 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:54:23.080428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:54:23.091375 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:54:23.092815 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:54:23.103039 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1457/2900] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:54:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:54:29.257417 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=26 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:54:33.098259 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=9 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:54:38.931178 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:54:38.940699 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553306-6be8-4164-985f-79a51109a13f service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:54:38.944316 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:54:38.944901 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:54:38.945182 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:54:38.945453 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:54:38.954998 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:54:38.956498 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:54:38.967030 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1444/2901] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:54:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:54:48.961920 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:56:43.672087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:56:43.673533 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:56:43.742047 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073325-a98e-480d-b801-66124b82fe7c service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:56:43.747023 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c5043-340b-4e50-9a82-52f43c9a5eb8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:56:43.747609 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c5043-340b-4e50-9a82-52f43c9a5eb8 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:56:43.748568 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c5043-340b-4e50-9a82-52f43c9a5eb8 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:56:43.748760 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c5043-340b-4e50-9a82-52f43c9a5eb8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:56:43.750490 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ad1c5043-340b-4e50-9a82-52f43c9a5eb8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:56:43.761654 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c5043-340b-4e50-9a82-52f43c9a5eb8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:56:43.763235 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c5043-340b-4e50-9a82-52f43c9a5eb8 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:56:43.774478 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1458/2902] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:56:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:56:53.770167 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=32 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:56:59.570672 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:56:59.572817 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:56:59.589421 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ce0f-b1ef-4b77-8471-b3bd70bebf4d service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:56:59.595467 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:56:59.596068 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:56:59.596350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:56:59.596623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:56:59.598370 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:56:59.611209 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:56:59.613000 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:56:59.624256 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1445/2903] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:56:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:57:01.196941 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2b515-8b69-4c2b-af20-bd241755b9d8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:57:01.197406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2b515-8b69-4c2b-af20-bd241755b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:57:01.197522 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2b515-8b69-4c2b-af20-bd241755b9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:57:01.216403 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8fa2b515-8b69-4c2b-af20-bd241755b9d8 None None] MFA Rules not processed for user `74a5ed7c176f44849677acf29c330e97`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:57:01.217606 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8fa2b515-8b69-4c2b-af20-bd241755b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:57:01.224920 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1459/2904] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:57:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:57:01.535948 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:57:01.537376 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:57:01.549492 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f2ab9-e5b1-4664-902a-735a17937482 service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:57:01.553352 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:57:01.553943 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:57:01.554279 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:57:01.554891 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:57:01.556623 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:57:01.567154 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:57:01.568588 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:57:01.576513 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1446/2905] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:57:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:57:02.254614 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2b515-8b69-4c2b-af20-bd241755b9d8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:57:02.268140 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2b515-8b69-4c2b-af20-bd241755b9d8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:57:02.273233 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:57:02.273880 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:57:02.274161 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:57:02.274460 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:57:02.276272 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:57:02.288629 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:57:02.290197 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:57:02.301100 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1460/2906] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:57:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:57:11.572877 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=12 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:57:12.295996 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=9 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:57:16.724954 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:57:16.738124 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f82125-7775-4dcf-921c-ba8c74e595d7 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:57:16.743581 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59551c2-8238-4db4-886e-d7c07aeed5e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:57:16.744167 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59551c2-8238-4db4-886e-d7c07aeed5e2 service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:57:16.744442 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59551c2-8238-4db4-886e-d7c07aeed5e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:57:16.744743 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59551c2-8238-4db4-886e-d7c07aeed5e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:57:16.746451 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-c59551c2-8238-4db4-886e-d7c07aeed5e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:57:16.759190 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59551c2-8238-4db4-886e-d7c07aeed5e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:57:16.760734 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59551c2-8238-4db4-886e-d7c07aeed5e2 service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:57:16.771778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1447/2907] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:57:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:57:26.766727 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:59:14.152428 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:59:14.153831 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:59:14.209392 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13516828-d3db-47ba-9cf2-7fd1394c4e3d service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:59:14.212905 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:59:14.213487 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:59:14.213761 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:59:14.214027 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:59:14.222697 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:59:14.224099 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:59:14.239992 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1461/2908] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:59:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:59:24.228901 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=26 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:59:26.787835 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb3824-b670-4249-bf61-2c966bc5c659 None None] REQUEST_METHOD: `POST` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:59:26.788345 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb3824-b670-4249-bf61-2c966bc5c659 None None] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:59:26.788463 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb3824-b670-4249-bf61-2c966bc5c659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:59:26.807813 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.auth.core [None req-cfbb3824-b670-4249-bf61-2c966bc5c659 None None] MFA Rules not processed for user `74a5ed7c176f44849677acf29c330e97`. Rule list: `[]` (Enabled: `True`). {{(pid=80267) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:59:26.809637 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-cfbb3824-b670-4249-bf61-2c966bc5c659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:59:26.816505 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1448/2909] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 02:59:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:59:26.855704 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:59:26.857196 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:59:26.868494 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cdd89-6948-40a3-a372-e796a656e5dc service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:59:26.872889 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1149adba4d37480b8d4fee34c77d340d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:59:26.878001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:59:26.878001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:59:26.878001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:59:26.878001 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:59:26.889641 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:59:26.891118 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:59:26.902094 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1462/2910] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:59:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:59:27.184543 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb3824-b670-4249-bf61-2c966bc5c659 None None] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:59:27.194248 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb3824-b670-4249-bf61-2c966bc5c659 None None] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:59:27.197778 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:59:27.198386 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:59:27.198667 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:59:27.198993 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:59:27.206448 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:59:27.207871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:59:27.216424 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1449/2911] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:59:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:59:27.953569 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:59:27.963033 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19d5d4-2731-45e8-a349-9fd3a2cee85b service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:59:27.966925 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:59:27.967511 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:59:27.967789 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:59:27.968059 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:59:27.969850 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:59:27.983963 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:59:27.985239 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:59:27.997082 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1463/2912] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:59:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:59:30.497258 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:59:30.506200 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fce25-2227-4fea-9ae8-a2dde6627ae6 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:59:30.510193 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:59:30.510947 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:59:30.511392 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:59:30.511761 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:59:30.519459 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:59:30.520871 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:59:30.528724 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1450/2913] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:59:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:59:37.991384 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=12 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:59:40.525107 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=15 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:59:44.509329 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:59:44.518705 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b85da-014d-4421-bf9f-37450e82d864 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:59:44.522306 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:59:44.522917 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:59:44.523194 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:59:44.523461 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:59:44.533589 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:59:44.535058 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:59:44.545369 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1464/2914] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 02:59:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:59:54.540170 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 03:01:37.564551 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:37.579651 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2893e2-6d36-48d8-b7e3-e81fea313971 service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:37.586902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4cfb26c57514e2cbaa73cb036118dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:37.587628 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:37.587878 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:37.588291 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:37.590104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:01:37.604104 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:37.605725 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:37.617514 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1451/2915] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 03:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 03:01:46.931799 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:46.933200 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:01:47.017102 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa27563-817b-46ea-be7c-4326d795ba83 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:47.022813 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:47.023409 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:47.023684 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:47.023951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:47.032866 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:47.035426 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:47.045361 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1465/2916] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 03:01:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 03:01:47.612221 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=6 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 03:01:50.892232 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:50.909584 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862c046-94e1-4714-9bd7-a42ac24c682a service nova] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:50.913185 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8839af749e44fac8a7a311b61463c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:50.913780 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:50.914058 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:50.914966 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:50.975212 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:50.976638 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:50.987171 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1452/2917] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 03:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 03:01:54.436406 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:54.452189 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999d638-7867-45e6-a44b-da45e089f6d8 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:54.458106 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:54.458755 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:54.459015 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:54.459287 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:54.461060 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:01:54.474087 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:54.475773 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:54.487657 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1466/2918] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 03:01:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 03:01:57.902542 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:57.925024 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4a849-90ca-4777-a7a1-85ef443a677f service neutron] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:57.931169 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:57.933529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] REQUEST_METHOD: `GET` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:57.933529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:57.933529 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:57.935405 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:01:57.950460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:57.952545 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:57.964189 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1453/2919] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 03:01:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 03:01:58.031336 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:58.061937 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b6d3-9bc6-4432-881b-2942c68d7fc4 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:58.067560 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:58.068181 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:58.068456 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:58.068752 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] PATH_INFO: `/v3/users/74a5ed7c176f44849677acf29c330e97` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:58.069870 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a5ed7c176f44849677acf29c330e97)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:58.071823 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:58.227355 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a5ed7c176f44849677acf29c330e97'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.287529 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a5ed7c176f44849677acf29c330e97'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.300939 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a5ed7c176f44849677acf29c330e97'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.301578 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a5ed7c176f44849677acf29c330e97'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.302184 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a5ed7c176f44849677acf29c330e97'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.315143 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1467/2920] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 03:01:58 2022] DELETE /identity/v3/users/74a5ed7c176f44849677acf29c330e97 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 03:01:58.586462 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:58.599533 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de857ef6-d4aa-4b3e-b772-e23a794a884b service placement] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:58.656750 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:58.657369 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:58.657648 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:58.658569 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] PATH_INFO: `/v3/projects/cdf9c8d3eace468d852b873213f392f4` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:58.659836 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf9c8d3eace468d852b873213f392f4)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:58.661742 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:58.768350 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf9c8d3eace468d852b873213f392f4', 'request_id': 'req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.774483 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf9c8d3eace468d852b873213f392f4', 'request_id': 'req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.824566 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf9c8d3eace468d852b873213f392f4', 'request_id': 'req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.835416 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 03:01:58.848165 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] The token cache is being invalidate because project cdf9c8d3eace468d852b873213f392f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 03:01:58.848454 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.849073 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:58.850556 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1454/2921] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 03:01:58 2022] DELETE /identity/v3/projects/cdf9c8d3eace468d852b873213f392f4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 03:01:58.855034 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:58.856146 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:01:58.869951 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c110028-6d18-4fce-8ea0-0055cef1252a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:58.888899 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 03:01:58.931694 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:58.932324 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:58.932604 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:58.932914 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] PATH_INFO: `/v3/users/5a998830266e4440869f918cb78f4221` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:58.934003 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a998830266e4440869f918cb78f4221)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:58.935952 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:59.118974 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a998830266e4440869f918cb78f4221'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.195845 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a998830266e4440869f918cb78f4221'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.208495 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a998830266e4440869f918cb78f4221'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.209133 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a998830266e4440869f918cb78f4221'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.209734 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a998830266e4440869f918cb78f4221'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.222402 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1468/2922] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 03:01:58 2022] DELETE /identity/v3/users/5a998830266e4440869f918cb78f4221 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 03:01:59.457457 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Authenticating user token {{(pid=80267) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:01:59.459760 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.fernet_utils [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80267) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:01:59.476403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0f3e8-2916-4d9c-84f0-544a44d21ed7 admin admin] Validating token access rules against request {{(pid=80267) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:01:59.535991 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5565152ee164c0da536fef7f9cd3d6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e036fca31f44e57a9a6f1c124d31c6b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80267) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:01:59.536620 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:01:59.536902 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:01:59.537177 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] PATH_INFO: `/v3/projects/90c196e44a4e40a49af4b5e27e3cceb1` {{(pid=80267) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:01:59.538403 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c196e44a4e40a49af4b5e27e3cceb1)` {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:01:59.540313 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] RBAC: Authorization granted {{(pid=80267) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:01:59.624748 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c196e44a4e40a49af4b5e27e3cceb1', 'request_id': 'req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.631013 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c196e44a4e40a49af4b5e27e3cceb1', 'request_id': 'req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.687619 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c196e44a4e40a49af4b5e27e3cceb1', 'request_id': 'req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9'} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.698460 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] The token cache is being invalidate because project 90c196e44a4e40a49af4b5e27e3cceb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80267) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 03:01:59.698697 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.699330 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG keystone.notifications [None req-c3196d95-2cbd-4db0-a155-5b4c8589b6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80267) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 03:01:59.700802 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: [pid: 80267|app: 0|req: 1455/2923] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 03:01:59 2022] DELETE /identity/v3/projects/90c196e44a4e40a49af4b5e27e3cceb1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 03:02:09.218670 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 03:02:09.695394 dv-cinder-852261-2-iscsi devstack@keystone.service[80267]: DEBUG dbcounter [-] [80267] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80267) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 03:06:50.590203 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 03:06:50.591660 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:06:50.664070 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32606df9-ee09-460b-9cf7-448fe79620f2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 03:06:50.723477 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa037eed-a35f-45e4-9936-47b280548f2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8547ea522755494ba5177d120561dcff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94e3fa8dc6e745569f8ffb7b76448278', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 03:06:50.724101 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa037eed-a35f-45e4-9936-47b280548f2c service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 03:06:50.724381 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa037eed-a35f-45e4-9936-47b280548f2c service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 03:06:50.724656 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa037eed-a35f-45e4-9936-47b280548f2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 03:06:50.726407 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fa037eed-a35f-45e4-9936-47b280548f2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 03:06:50.737935 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa037eed-a35f-45e4-9936-47b280548f2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 03:06:50.739471 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa037eed-a35f-45e4-9936-47b280548f2c service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 03:06:50.749468 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1469/2924] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 03:06:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 03:07:00.745149 dv-cinder-852261-2-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=58 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}